Title of Invention	ARRANGEMENT AND METHOD FOR CONTROLLING A COMPUTER APPARATUS ASSOCIATED WITH A GRAPHICAL DISPLAY
Abstract	The present invention relates to a computer based eye-tracking solution. A computer apparatus is associated with a graphical display, which in turn, represents at least one GUI-component (220a 220n) that is adapted to be manipulated based on user-generated commands. An event engine (210) is adapted to receive an eye-tracking data signal (DEYE) that describes a user's point of regard on the display. Based on the signal (DEYE), the event engine (210) is adapted to produce a set of non-cursor controlling event output signals (D-HIi), which influence the at least one GUI-component (220a, ..., 220n). Each non-cursor controlling event output signal (D-HIi) describes a particular aspect of the user's ocular activity in respect of the display. Initially, the proposed event engine (210) receives a control signal request (Ra,... , Rn) from each of the at least one GUI-component (220a,..., 220n). The control signal request (Ra, ... , Rn) defines a sub-set of the set of non-cursor controlling event output signals (D-HIi) which is required by the particular GUI-component (220a, .... 220n). The event engine (210) delivers non-cursor controlling event output signals (D-HIi) to the at least one GUI-component (220a,..., 220n) in accordance with each respective control signal request (Ra , Rn).

Title of Invention

ARRANGEMENT AND METHOD FOR CONTROLLING A COMPUTER APPARATUS ASSOCIATED WITH A GRAPHICAL DISPLAY

Abstract

The present invention relates to a computer based eye-tracking solution. A computer apparatus is associated with a graphical display, which in turn, represents at least one GUI-component (220a 220n) that is adapted to be manipulated based on user-generated commands. An event engine (210) is adapted to receive an eye-tracking data signal (DEYE) that describes a user's point of regard on the display. Based on the signal (DEYE), the event engine (210) is adapted to produce a set of non-cursor controlling event output signals (D-HIi), which influence the at least one GUI-component (220a, ..., 220n). Each non-cursor controlling event output signal (D-HIi) describes a particular aspect of the user's ocular activity in respect of the display. Initially, the proposed event engine (210) receives a control signal request (Ra,... , Rn) from each of the at least one GUI-component (220a,..., 220n). The control signal request (Ra, ... , Rn) defines a sub-set of the set of non-cursor controlling event output signals (D-HIi) which is required by the particular GUI-component (220a, .... 220n). The event engine (210) delivers non-cursor controlling event output signals (D-HIi) to the at least one GUI-component (220a,..., 220n) in accordance with each respective control signal request (Ra , Rn).

Full Text	THE BACKGROUND OF THE INVENTION AND PRIOR ART: The present invention relates generally to computer based eye-tracking systems. More particularly the invention relates to an arrangement for controlling a computer apparatus associated with a graphical display and a corresponding method which is being performed by a computer readable medium having a program recorded therein. The human computer interaction was revolutionized by the introduction of the graphical user interface (GUI). Namely, thereby, an efficient means was provided for presenting information to a user with a bandwidth which immensely exceeded any prior channels. Over the years the speed at which information can be presented has increased further through color screens, enlarged displays, intelligent graphical objects (e.g. pop-up windows), window tabs, menus, toolbars and sounds. During this time, however, the input devices have remained essentially unchanged, i.e. the keyboard and the pointing device (e.g. mouse, track ball or touch pad). In recent years, handwriting devices have been introduced (e.g. in the form of a stylus or graphical pen). Nevertheless, while the output bandwidth has multiplied several times, the input ditto has been substantially unaltered. Consequently, a severe asymmetry in the communication band-width in the human computer interaction has occurred. In order to decrease this bandwidth gap, various attempts have been made to use eye-tracking devices. However, in many cases these devices miss the mark in one or several respects. One problem is that the prior-art solutions fail to take a holistic view on the input interfaces to the computer. Thereby, com- paratively heavy motor tasks may be imposed on the eyes, which in fact are strictly perceptive organs. Typically, this leads to fatigue symptoms and a degree of discomfort experienced by the user. This is particularly true if an eye tracker is used to control a cursor on a graphical display, and for various reasons the eye tracker fails to track the user's point of regard suffi- ciently well, so that there is a mismatch between the user's actual point of regard and the position against which the cursor is controlled. Instead of controlling the cursor directly, an eye gaze signal may be used to select an appropriate initial cursor position. The do- cument US, 6,204,828 discloses an integrated gaze / manual cursor positioning system, which aids an operator to position a cursor by integrating an eye-gaze signal and a manual input. When a mechanical activation of an operator device is detected the cursor is placed at an initial position which is predetermined with respect to the operator's current gaze area. Thus, a user- friendly cursor function is accomplished. The document US, 6,401,050 describes a visual interaction sys- tem for a shipboard watch station. Here, an eye-tracking camera monitors an operator's visual scan, gaze location, dwell time, blink rate and pupil size to determine whether additional cueing of the operator should be made to direct the operator's attention to an important object on the screen. The document US, 5,649,061 discloses a device for estimating a mental decision to select a visual cue from a viewer's eye fixation and corresponding event evoked cerebral potential. An eye tracker registers a viewing direction, and based thereon fixation properties may be determined in terms of duration, start and end pupil sizes, saccades and blinks. A corresponding single event evoked cerebral potential is extracted, and an artificial neural network estimates a selection interest in the gaze point of regard. After training the artificial neural network, the device may then be used to control a computer, such that icons on a display are activated according to a user's estimated intentions without requiring any manipulation by means of the user's hands. A few attempts have also been made to abstract user-generated input data into high-level information for controlling a computer. For example, the document US 2004/0001100 describes a multimode user interface, where a flexible processing of a user input is made possible without having to switch manually between different input modes. Instead, within the data streams different information categories are distinguished depending on a context in which the data streams are generated. Although this strategy may indeed enhance the efficiency of the man-machine interaction, no multimodal solution has yet been presented according to which eye-tracking data is processed optimally. On the contrary, with only very few exceptions, to- day's eye tracking interfaces are each tailored for one specific task only. Thus, any processing of eye tracking data in respect of a first application cannot be reused by a second application, and vice versa. Hence, if multiple eye-controlled applications are used in a single computer, one particular eye-tracking data processing unit is typically required for each application. Natu- rally, in such a case, there is a high risk that the different appli- cations perform a substantial amount of overlapping eye-trac- king data processing. Moreover, each designer of eye-control- lable applications needs to have expertise in both eye tracking technology and the interpretation of eye-tracking data in order to extract the data required by the application. SUMMARY OF THE INVENTION The object of the present invention is therefore to provide a ho- listic means of controlling a computer apparatus based on a user's ocular activity, which alleviates the above problems and thus offers an efficient man-machine interaction with a minimal amount of double processing, i.e. wherein as much as possible of the eye-tracking data processing is performed centrally in respect of GUI-components that may belong to two or more se- parate applications. According to one aspect of the invention, the object is achieved by the arrangement as initially described, wherein the event engine is adapted to receive a control signal request from each of the at least one GUI-component. The control signal request defines a sub-set of the set of non-cursor controlling event output signals, which is required by the GUI-component in question. The event engine is also adapted to deliver non-cursor controlling event output signals to the at least one GUI-compo- nent in accordance with each respective control signal request. This arrangement is advantageous because thereby a very flexible interface is attained towards any applications which are controllable by means of eye tracking signals. This, in turn, enables software developers without an in-depth knowledge of eye-tracking technology to design eye-controllable applications. Therefore, the invention is believed to stimulate the develop- ment of new such applications, and consequently render further improvements of the human computer interaction possible. Moreover, the computer's processing resources are freed for alternative purposes, since a high-level eye-tracking data signal derived in respect of one application may be reused by one or more additional applications. In other words, the event engine is adapted to perform a centralized eye-tracking data processing for GUI-components (or potential "subscribers") on-demand. Thus, each of these components and their requested control signals need not be explicitly known at the point in time when the event engine is implemented. Instead, according to the invention, one or more GUI-components may request relevant control signals in connection with the implementation of a later added eye-controllable application. Of course, the opposite may also be true, namely that a set of control signal requests is presented once and for all at start-up of the proposed arrange- ment. According to a preferred embodiment of this aspect of the inven- tion, the computer apparatus is also adapted to receive a cursor control signal, and in response to the cursor control signal, control a graphical pointer on the display. Thus, for instance, the event engine and the cursor control signal may interact jointly with GUI-components represented on the display, such that a very intuitive man-machine interaction is accomplished. According to another preferred embodiment of this aspect of the invention, at least one GUI-component is adapted to generate at least one respective output control signal upon a user mani- pulation of the component. This means that by manipulating a GUI-component, the user may cause the computer apparatus to generate outgoing signals to one or more external units, such as a printer, a camera etc. Naturally, this is a highly desirable feature. Moreover, the fact that one or more GUI-components may generate an output control signal does not preclude that with respect to one or more other GUI-components, the non- cursor controlling event output signals may exclusively affect the component internally. According to a preferred embodiment of this aspect of the inven- tion, the event engine is adapted to produce at least a first signal of the non-cursor controlling event output signals based on a dynamic development of the eye-tracking data signal. Thereby, a time parameter of the user's ocular activity may be used to control functions and processes of the computer appa- ratus. For instance, the time parameter may reflect a dwell time of the user's gaze within a particular region on the display, identify a certain gaze pattern etc. Many types of advanced eye- controllable functions can thereby be realized. According to yet another preferred embodiment of this aspect of the invention, at least one GUI-component is adapted to interpret a non-cursor controlling event output signal as a rep- resentation of the user's intention. In response to this estimated intention, a user manipulation of the component is triggered. For example this may involve activating one or more computer functions based on a command history. This is advantageous because thereby the command input procedure may be simpli- fied. According to still another preferred embodiment of this aspect of the invention, at least one GUI-component is adapted to interpret a non-cursor controlling event output signal as an esti- mated attention level of the user. In response to this estimated attention level, a user manipulation of the component is triggered. One advantage attainable thereby is that the com- puter's behavior may be adapted to match a current attention level of the user. According to yet another preferred embodiment of this aspect of the invention, at least one GUI-component is adapted to interpret a non-cursor controlling event signal as a state-of-mind parameter of the user, and in response thereto trigger a user manipulation of the component. This feature is desirable because it allows the computer to behave differently depending on whether the user appears to be focused/concentrated, distracted, tired/unfocused or confused etc. According to still another preferred embodiment of this aspect of the invention, the event engine is adapted to receive at least one auxiliary input signal, such as a signal from a button or a switch, a speech signal, a movement pattern of an input mem- ber, a camera registered gesture pattern or facial expression, or an EEG (electroencephalogram)-signal. On the further basis of this auxiliary signal the event engine produces the set of non- cursor controlling event output signals. Thereby, a highly efficient combination of user input signals may be used to control the computer. Generally, these combinations can be made very intuitive and easy to learn. According to another aspect of the invention the object is achieved by the method as initially described, wherein a control signal request is received from each of the at least one GUI- component. The control signal request defines a sub-set of the set of non-cursor controlling event output signals, which is required by the particular GUI-component. Non-cursor con- trolling event output signals are then delivered to the at least one GUI-component in accordance with each respective control signal request. The advantages of this method, as well as the preferred embodi- ments thereof, are apparent from the discussion hereinabove with reference to the proposed arrangement. According to a further aspect of the invention the object is achieved by a computer program, which is directly loadable into the internal memory of a computer, and includes software for controlling the above proposed method when said program is run on a computer. According to another aspect of the invention the object is achieved by a computer readable medium, having a program recorded thereon, where the program is to control a computer to perform the above proposed method. The invention dramatically increases the available bandwidth for transferring information from a user to a computer apparatus, i.e. essentially generating commands, however not necessarily perceived as such by the user. Therefore, this increase of the bandwidth places no additional cognitive workload on the user. On the contrary, by means of the invention, the cognitive workload may, in fact, be reduced. At the same time, the increased bandwidth vouches for an improved efficiency of the man-machine interaction. Moreover, by means of the invention, commands which traditio- nally have required hand and/or finger manipulations, can be efficiently and effortlessly effected based on the user's eye activity. Naturally, this is desirable in a broad range of appli- cations, from disabled computer users, support operators in a call-center environment (e.g. when entering/editing data in a customer relationship management application), users of advanced computer aided design (CAD) tools, surgeons, to drivers and pilots who, for various reasons, cannot effectively produce hand- and/or finger-based commands. Even in situations where the users have their hands and fingers readily accessible, the invention may be useful to improve the ergono- mics and the reduce risk of e.g. repetitive strain injuries. Alternatively, the environment in which the computer apparatus is placed may be so clean or dirty that either the environment has to be protected from possible emissions from the computer apparatus, or reverse this apparatus must be protected against hazardous substances in the environment and therefore has to be encapsulated to such a degree that a traditional entry of input of commands is made impossible, or at least impractical. The invention offers an excellent basis for developing new software and computer applications, which are controllable by the eyes of a user. Thereby, in the long term, the invention vouches for a deep and unconstrained integration of eye inter- action applications into the standard computer environment. Further advantages, advantageous features and applications of the present invention will be apparent from the following des- cription and the dependent claims. BRIEF DESCRIPTION OF THE DRAWINGS The present invention is now to be explained more closely by means - of preferred embodiments, which are disblosed as examples, and with reference to the attached drawings. DESCRIPTION OF PREFERRED EMBODIMENTS OF THE INVENTION Figure 1 shows an overview picture of a typical: use-case according to the invention. Here, a user 110 controls a computer apparatus 130, at least partly based on an eye-tracking data signal DEYE, which describes the user's 110 point of regard x, y on a display 120. Thus, by watching a representation of a GUI- component 220 on the display 120, the user 110 may generate commands to the computer apparatus 130. This manipulation is enabled, since the GUI-component 220 is adapted to be, at leasl indirectly, influenced by the eye-tracking data signal,DEYE. The invention presumes that the eye-tracking data signal DEYE may result in events, related to any task performable by the computer apparatus, apart from affecting a cursor/pointer on the display 120. It should be noted that according to the invention, any type of known computer screen or monitor, as well as combinations of two or more separate displays may represent the display 120. For example, the display 120 may constitute a pair of stereoscopic screens, a heads-up display (HUD), a head- mounted display (HMD) and a presentation means for a virtual environment, such as the eyepieces of a pair of 3D-glasses or a room where the walls include projection screens for presenting a virtual environment. Naturally, in order to produce the eye-tracking data signal DEYE, the display 120 is associated with, or includes, an eye-tracker. This unit is not a subject of the present patent application, and therefore will not be described in detail here. However, the eye- tracker is preferably embodied by the solution described in the Swedish patent application 0203457-7, filed on 21 November 2002 in the name of the applicant. Preferably, a graphics control signal C-GR is generated by the computer apparatus 130 to accomplish visual feedback infor- mation on the display 120. The visual feedback information is generated in response to any user-commands received by the computer apparatus 130, so as to confirm to the user 110 any commands, which are based on the eye-tracking data signal DEYE- Such a confirmation is especially desirable when the com- mands are produced by means of perceptive organs; for exam- ple the human eyes. According to a preferred embodiment of the invention, besides the eye-tracking data signal DEYE. the computer apparatus 130 is adapted to receive a cursor control signal K, which controls the position of a graphical pointer on the display 120. Of course, the graphics control signal C-GR may be based also on the cursor control signal K. Figure 2 shows an arrangement according to an embodiment of the invention, which may be realized by means of the computer apparatus 130 described above with reference to the figure 1. The arrangement includes an event engine 210 and at least one GUI-component 220, which is adapted to be manipulated based on user-generated commands, at least partly expressed by the eye-tracking data signal DEYE- The event engine 210 is adapted to receive the eye-tracking data signal DEYE. and based thereon produce a set of non-cursor controlling event output signals D- Hlj that influence the at least one GUI-component 220. Each non-cursor controlling event output signal D-Hlj, in turn, descri- bes a particular aspect of the user's 110 ocular activity in respect of the display 120. For example, a first signal may indicate whether the user's 110 gaze is directed towards the display at all (i.e. a "gaze-on- display" signal), a second non-cursor controlling event output signal may reflect a dwell time of the user's 110 gaze within a- certain area on the display 120, a third signal may designate a gaze fixation (at a specific point), a fourth signal may indicate whether the gaze saccades, a fifth signal may indicate whether the gaze follows a smooth path, a sixth signal may reflect that the user 110 reads a text, and a seventh signal may be triggered if the user 110 appears to be distracted, based on the particular eye-tracking data signal DEYE that he/she produces. According to the invention, the event engine 210 receives a respective control signal request Ra, ..., Rn from each of the at least one GUI-component 220a, ..., 220n. The control signal request, say Ra, defines a sub-set of the set of non-cursor con- trolling event output signals D-Hli which are required by the par- ticular GUI-component, say 220a, to operate as intended. The event engine 210 then delivers non-cursor controlling event output signals D-Hli to each of the at least one GUI-component 220a, ..., 220n in accordance with the respective control signal request Ra, ..., Rn. A most efficient processing is accomplished if the event engine 210 exclusively produces those event output signals D-Hli which are actually requested by at least one GUI-component. However, according to the invention, it is also conceivable that all non-cursor controlling event output signals D-Hli that are possible to produce are always generated by the event engine 210, irrespective of whether a corresponding control signal request has been received or not. Namely, this simplifies the procedure, and depending on the application, this strategy may not require an overly extensive processing. According to a preferred embodiment of the invention, each GUI-component 220a, .... 220n is adapted to generate at least one respective output control signal Ca, ..., Ca upon a user manipulation of the component 220a, ..., 220n. Thus, in addition to generating the above-mentioned visual feedback information, one or more internal or peripheral devices may be influenced by means of the output control signals Ca, ..., Ca. For example a print job may be initiated, a computing task may be executed, an e-mail may be sent out, a camera may be triggered to take a picture etc. As mentioned above, the non-cursor controlling event output signals D-HIi may describe many different aspects of the eye- tracking data signal DEYE- According to one embodiment of the invention, at least one output signal D-Hli is based on a dynamic development of the eye-tracking data signal DEYE- Thereby, the signal can represent a particular gaze pattern over the display 120. The gaze pattern, in turn, may be determined, for instance to constitute saccades, a smooth pursuit, periods of fixation or reading. A non-cursor controlling event output signal D-Hli may also indicate gaze-enter/gaze-leave data. This data is a parameter which reflects the time instances when the eye-tracking data signal DEYE indicates that the user's point of regard falls within a GUI component's representation on the display. Hence, the gaze-enter data is generated when the user's gaze falls onto a GUI component's representation on the display, and the gaze- leave data is generated when the gaze is directed outside this representation. The above-mentioned dwell time for a GUI-component is typi- cally defined as the period between the gaze-enter data and the gaze-leave data with respect to a particular GUI component. It is normally preferable to link an activation signal to the dwell time, such that for instance an "eye-button" is activated when a certain dwell time is reached. Alternatively, a button, a switch, a speech signal, a movement pattern of an input member, a camera registered gesture pattern or facial expression may constitute the activation signal. Moreover, an eye blink or a pre- defined EEG-signal may cause the activation signal. However, the latter types of signals are usually relatively difficult for the user to control with a sufficient accuracy. According to one embodiment of the invention, the event engine 210 is adapted to receive at least one auxiliary input signal DJ, and produce the set of non-cursor controlling event output signals D-Hli on the further basis of this signal. The auxiliary input signal DJ may originate from a button, a switch, a speech signal, a movement pattern of an input member, a camera registered gesture pattern or a facial expression, or an EEG- signal. Thereby, based on combinations of the eye-tracking data signal DEYE and one or more auxiliary input signals DJ composite user commands may be created, which are very intuitive and easy to learn. Hence, a highly efficient man-machine interaction with the computer apparatus 130 may be accomplished. For instance, watching an eye-button for the document and uttering the control word "open" can open a text document. If a more complex speech recognition is available, an Internet search may be effected by focusing the gaze towards a relevant text input box while uttering the desired search terms, and so on. According to one embodiment of the invention, at least one GUI- component 220a, ..., 220n is adapted to'interpret a non-cursor controlling event output signal D-Hli from the event engine 210 as an estimated intention of the user 110. Then, in response to the estimated intention, a user manipulation of the component 220a, ..., 220n is triggered. Preferably, the event engine 210 estimates a user intention based on multiple input sources received as auxiliary input signals DJ, which may include a key- board signal, a mouse signal, voice data and camera images. However, naturally, the eye-tracking data signal DEYE may also constitute a basis for the estimated user intention. For example, important information can be drawn from different gaze patterns and fixation times. According to another embodiment of the invention, at least one GUI-component 220a, ..., 220n is adapted to interpret a non- cursor controlling event output signal D-Hli from the event engine 210 as an estimated attention level of the user 110. Correspondingly, a user manipulation of the component 220a, ..., 220n is triggered in response to the estimated attention level. Also the attention level may be estimated based on the auxiliary input signals DJ, for instance originating from the keyboard, the mouse, voice data and camera images, and the eye-tracking data signal DEYE- Particularly, gaze patterns, fixation points and fixation times constitute an important basis for determining the user's 110 attention level. Preferably, the GUI-components 220a, ..., 220n vary their behavior depending on the estimated attention level, such that the components' characteristics match the user's 110 current performance. Furthermore, according to one embodiment of the invention, at least one GUI-component 220a, .... 220n is adapted to interpret a non-cursor controlling event output signal D-Hli from the event engine 210 as a state-of-mind parameter of the user 110. The state-of-mind parameter reflects a general user 110 status, for example whether he/she appears to be focused/concentrated, distracted, tired/unfocused or confused. For example, the state- of-mind parameter may indicate an approximated 20% degree of tiredness and an approximated 50% degree of attention. Then, based on the estimated state-of-mind, a user manipulation of the component 220a, ..., 220n is triggered. Typically, the number of and the contents of any help menus and pop-up windows may be adapted in response to the estimated state-of-mind. How- ever, in operator environments, such as radar watch stations where the attention level as well as the state-of-mind may be truly critical, the security may be improved by pointing out targets etc. that the operator has not yet observed. According to one embodiment of the invention, the event engine 210 is associated with a template library 230, which contains generic GUI-components, such as eye-buttons, scroll bars, multiview toolbars (see below with reference to figure 4), text input fields (see below with reference to figure 5), expandable text input fields (see below with reference to figure 6) and scroll windows (see below with reference to figure 7). Thereby, by means of the template library 230, a software designer may conveniently create functions and controls which can be manipulated based on a user's ocular activity in respect of a display. Of course, after completing the design of a particular eye-controllable application, however, the template library 230 has no actual function in respect of the GUI-components that may originate from its generic components. Nevertheless, the template library 230 may again become useful in case of a future upgrade or a redesign of the application. According to the invention, it is not technically necessary that any GUI-component manipulations be graphically confirmed on the display. However, this is generally preferable from a user- friendliness point-of-view. Figure 3a shows a schematic symbol 310, which represents an eye-controllable GUI-component on a display that is set in a non-observed mode (i.e. the eye-tracking data signal DEYE indicates that the user's point of regard lies outside the symbol 310). Figure 3b shows the symbol 310 in an observed mode, which is, set when the eye-tracking data signal DEYE indicates that the user's point of regard falls within the display area represented by the symbol 310. In the observed mode, the symbol 310 contains a centrally located object 311. This object 311 confirms to the user that the computer apparatus has registered that his/her gaze presently is directed towards the symbol 310. Thus, any manipulations in respect of the GUI-component associated with the symbol 310 can be performed. An important advantage attained by means of the centrally located object 311 is that this object assists the user in focusing his/her gaze to the center of the symbol 310. Thereby, a more reliable eye-tracking function is accomplished, and for a given eye-tracker, the symbols 310 can be made smaller than otherwise. Of course, the symbol 310 and the centrally located object 311 may have any other outline than the square repre- sentation of figures 3a and 3b. Moreover, to further improve the visual cueing, the object 311 may be animated and or have a particularly interesting color or shape. Figure 4 illustrates a first embodiment according to the inven- tion, where a proposed multiview toolbar 401 is used to control applications in a frame 400. The multiview toolbar 401 here includes four different eye-buttons, which each may contain a thumbnail image (not shown) of a respective application with instance as shown in the figure 6b. Hence, more information than what was initially visible in the field 620 may be shown. The thus expanded text field 620 may even cover graphical objects which otherwise are shown in the frame 400. This is ad- vantageous, because thereby the information may be presented on-demand, so that the frame 400 and the sub-frame 520 contain more data than what actually can be fitted therein. For instance, a text scrolling, which otherwise would have been necessary can be avoided. Figure 7 illustrates a fourth embodiment according to the invention, which realizes a scrolling function based on a user's ocular activity. Here, a third application, e.g. a map viewer, is associated with a third button 710. Thus, by watching the third button 710, and activating its associated GUI-component, either based on a gaze dwell time or by means of a separate activation signal, the computer apparatus opens up a map sub-frame 720 within the frame 400. This sub-frame, in turn, presents a digi- tized map. It is here presumed that the map is larger than the available presentation area in the map sub-frame 720, so that only a part of the map can be presented at the same time. A scrolling with- respect to the map is achieved based on the user's point or regard. Preferably, no scrolling occurs as long as the eye-tracking data signal indicates that the point of regard lies within a central area delimited by a first dashed line a, a second dashed line b, a third dashed line c and a fourth dashed line d, neither of which preferably are visible on the display. If however, the user's point of regard is placed outside any of the lines a, b, c or d, the map scrolls in a particular direction given by the point of regard. Specifically, this means that a point of regard below the line c results in a downward scroll along the arrow S; a point of regard above the line d results in a upward scroll along the arrow N; a point of regard to the right of the line b results in a rightward scroll along the arrow E; and a point of regard to the left of the line a results in a leftward scroll along the arrow W. Moreover, a point of regard, which lies below the line c and to the right of the line b results in a diagonal scroll along the arrow SE; a point of regard, which lies above the line d and to the right of the line b results in a diagonal scroll along the arrow NE; a point of regard, which lies above the line d and to the left of the line a results in a diagonal scroll along the arrow NW; and a point of regard, which lies below the line c and to the left of the line a results in a diagonal scroll along the arrow SW. This scroll function may either be activated based on a dwell time, or a separate activation signal may be required, such as a clicking a key/button or holding down a key/button. Furthermore, the scroll speed may depend on the distance between the point of regard and the respective lines a, b, c and d, such that a relatively long distance corresponds to a compa- ratively high speed, and vice versa. The scroll speed may also depend on the scroll time, and/or a length of the latest saccades registered in the eye-tracking data signal. Preferably, a maxi- mum scroll speed is set to such a value that the scrolled infor- mation is visible to the user at all possible speeds. According to one embodiment of the invention, the scroll function is stopped by pressing a key/button, releasing a key/button, the length of the latest saccades exceeding a particular value, the point of regard moving outside the map sub-frame 720, the point of regard moving towards the center of, the map sub-frame 720, or towards an opposite scroll activation line a, b, c, or d. It should be noted that, eyetrack-driven scrolling solutions as such are described in the prior art, for instance in the document US, 5,850,221. Here, a-page oriented or continuous information scrolling function is initiated or controlled based on where a viewer's eyes are looking. To sum up, the general method of controlling a computer appa- ratus according to the invention will now be described with reference to the flow diagram in figure 8. An initial step 810 receives a control signal request from each of at least one GUI-component, which is adapted to be manipu- lated based on user-generated, eye commands. The control signal request defines a sub-set of the set of non-cursor con- trolling event output signals, which is required by the particular GUI-component in order to operate as intended. A step 820 then receives an eye-tracking data signal, which des- cribes a user's point of regard on a display associated with a computer that embodies the at least one GUI-component. Subsequently, a step 830 produces a set of non-cursor con- trolling event output signals based on the eye-tracking data signal, plus any auxiliary input signal. Subsequently, a step 840 delivers the non-cursor controlling event output signals to the at least one GUI-component so that a relevant influence thereof may occur (i.e. according to each respective control signal request). It is presumed that each non- cursor controlling event output signal describes a particular aspect of the user's ocular activity in respect of the display. Thus, the non-cursor controlling event output signals express user-generated eye commands. After the step .840, the pro- cedure loops back to the step 820 for receiving an updated eye tracking data signal. All of the process steps, as well as any sub-sequence of steps, described with reference to the figure 8 above may be controlled by means of a programmed computer apparatus. Moreover, although the embodiments of the invention described above with reference to the drawings comprise computer apparatus and processes performed in computer apparatus, the invention thus also extends to computer programs, particularly computer programs on or in a carrier, adapted for putting the invention into practice. The program may be in the form of source code, object code, a code intermediate source and object code such as in partially compiled form, or in any other form suitable for use in the implementation of the process according to the invention. The program may either be a part of an operating system, or be a separate application. The carrier may be any entity or device capable of carrying the program. For example, the carrier may comprise a storage medium, such as a Flash memory, a ROM (Read Only Memory), for example a CD (Compact Disc) or a semiconductor ROM, an EPROM (Erasable Programmable Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), or a magnetic recording medium, for example a floppy disc or hard disc. Further, the carrier may be a transmissible carrier such as an electrical or optical signal which may be conveyed via electrical or optical cable or by radio or by other means. When the program is embodied in a signal which may be conveyed directly by a cable or other device or means, the carrier may be constituted by such cable or device or means. Alternatively, the carrier may be an integrated circuit in which the program is embedded, the integrated circuit being adapted for performing, or for use in the performance of, the relevant processes. The term "comprises/comprising" when used in this specification is taken to specify the presence of stated features, integers, steps or components. However, the term does not preclude the presence or addition of one or more additional features, integers, steps or components or groups thereof. The invention is not restricted to the described embodiments in the figures, but may be varied freely within the scope of the claims. We Claim: 1. An arrangement for controlling a computer apparatus (130) associated with a graphical display (120), the display (120) presenting a representation of at least one GUI-component (220) which is adapted to be manipulated based on user-generated commands, and at least one of the at least one GUI-component (220a,... 220n) is adapted to generate at least one respective output control signal (Ca ...Ca) upon a user manipulation of the component (220a,... 220n), the arrangement comprising an event engine (210) adapted to receive an eye-tracking data signal (DEYE) describing a user's (110) point of regard (x, y) on the display (120), and at least based on the eye-tracking data signal (DEYE) produce a set of non-cursor controlling event output signals (D-HIi) influencing the at least one GUI-component (220), each non-cursor controlling event output signal (D-HIi) describing a particular aspect of the user's (110) ocular activity in respect of the display (120), characterized in that the event engine (210) is adapted to: receive a control signal request (Ra, ..., Rn) from each of the at least one GUI- component (220a, ..., 220n), the control signal request (Ra, ..., Rn) defining a sub-set of the set of non-cursor controlling event output signals (D-HIi) which is required by the GUI-component (220a, ..., 220n) to operate as intended to produce the event output signals (D-HIi) which are requested by the at least one GUI-component (220a,...,220n) in the control signal request (Ra,...,Rn), and deliver non-cursor controlling event output signals (D-HIi) to the at least one GUI-component (220a, ..., 220n) in accordance with each respective control signal request (Ra, ..., Rn). 2. An arrangement as claimed in claim 1, wherein the computer apparatus (130) is adapted to: receive a cursor control signal (K), and control a graphical pointer on the display (120) in response to the cursor control signal (K). 3. An arrangement as claimed in any one of the preceding claims, wherein the event engine (210) is adapted to produce at least a first signal of the non-cursor controlling event output signals (D-HIi) based on a dynamic development of the eye-tracking data signal (DEYE). 4. An arrangement as claimed in claim 3, wherein the first signal represents a particular gaze pattern over the display (120). 5. An arrangement as claimed in claim 4, wherein at least one GUI- component (220a, ..., 220n) is adapted to interpret the first signal as an estimated intention of the user (110), and trigger a user manipulation of the component (220a,..., 220n) in response to the estimated intention. 6. An arrangement as claimed in any one of the claims 4 or 5, wherein at least one GUI-component (220a, ..., 220n) is adapted to interpret the first signal as an estimated attention level of the user (110), and trigger a user manipulation of the component (220a, ..., 220n) in response to the estimated attention level. 7. An arrangement as claimed in any one of the claims 4 to 6, wherein at least one GUI-component (220a, ..., 220n) is adapted to interpret the first signal as a state-of-mind parameter of the user (110), and trigger a user manipulation of the component (220a, ..., 220n) in response to the state- of-mind parameter. 8. An arrangement as claimed in any one of the preceding claims, wherein the event engine (210) is adapted to receive at least one auxiliary input signal (DJ) and produce the set of non-cursor controlling event output signals (D-HIi) on the further basis of the at least one auxiliary input signal (DJ). 9. An arrangement as claimed in any one of the preceding claims, wherein the at least one auxiliary input signal (DJ) is originated from at least one of a button, a switch, a speech signal, a movement pattern of an input member, a gesture pattern, a facial expression and an EEG-signal. 10. A method of controlling a computer apparatus (130) associated with graphical display (120), the display (120) representing at least one GUI- component (220) which is adapted to be manipulated based on user- generated commands, the method comprising: receiving an eye-tracking data signal (DEYE) which describes a user's (110) point of regard (x, y) on the display (120), and producing a set of non-cursor controlling event output signals (D-HIi) at least based on the eye-tracking data signal (DEYE), the set of non-cursor controlling event output signals (D-HIi) influencing the at least one GUI-component (220), and each non-cursor controlling event output signal (D-HIi) describing a particular aspect of the user's (110) ocular activity in respect of the display (120), the method characterized by receiving a control signal request (Ra, ... , Rn) from each of the at least one GUI-component (220a, ..., 220n), the control signal request (Ra, ..., Rn) defining a sub-set of the set of non-cursor controlling event output signals (D-HIi) which is required by the GUI-component (220a, ..., 220n) to operate as intended producing the event output signals (D-HIi) which are requested by the at least one GUI-component (220a,...,220n) in the control signal request (Ra,...,Rn), and delivering non-cursor controlling event output signals (D-HIi) to the at least one GUI-component (220a, ..., 220n) in accordance with each respective control signal request (Ra, ..., Rn). 11. A method as claimed in claim 10, comprising: receiving a cursor control signal (K), and controlling a graphical pointer on the display (120) in response to the cursor control signal (K). 12. A method as claimed in any one of the claims 10 or 11, comprising producing at least a first signal of the non-cursor controlling event output signals (D-HIi) based on a dynamic development of the eye-tracking data signal (DEYE). 13. A method as claimed in claim 12, comprising the first signal representing a particular gaze pattern over the display (120). 14. A method as claimed in claim 13, comprising at least one GUI-component (220a, ..., 220n) being adapted to interpret the first signal as an estimated intention of the user (110), and trigger a user manipulation of the component (220a,..., 220n) in response to the estimated intention. 15. A method as claimed in any one of the claims 13 or 14, comprising at least one GUI-component (220a, ..., 220n) being adapted to interpret the first signal as an estimated attention level of the user (110), and trigger a user manipulation of the component (220a, ..., 220n) in response to the estimated attention level. 16. A method as claimed in any one of the claims 13 to 15,comprising at least one GUI-component (220a, ... , 220n) being adapted to interpret the first signal as a state-of-mind parameter of the user (110), and trigger a user manipulation of the component (220a, ..., 220n) in response to the state- of-mind parameter. 17. A method as claimed in any one of the claims 11 to 18, comprising: receiving at least one auxiliary input signal (DJ), and producing the set of non-cursor controlling event output signals (D-HIi) on the further basis of the at least one auxiliary input signal (DJ). 18. A method as claimed in any one of the claims 10 to 17,comprising the at least one auxiliary input signal (DJ) being originated from at least one of a button, a switch, a speech signal, a movement pattern of an input member, a gesture pattern, a facial expression and an EEG-signal. (57) Abstract: The present invention relates to a computer based eye-tracking solution. A computer apparatus is associated with a graphical display, which in turn, represents at least one GUI-component (220a 220n) that is adapted to be manipulated based on user-generated commands. An event engine (210) is adapted to receive an eye-tracking data signal (DEYE) that describes a user's point of regard on the display. Based on the signal (DEYE), the event engine (210) is adapted to produce a set of non-cursor controlling event output signals (D-HIi), which influence the at least one GUI-component (220a, ..., 220n). Each non-cursor controlling event output signal (D-HIi) describes a particular aspect of the user's ocular activity in respect of the display. Initially, the proposed event engine (210) receives a control signal request (Ra,... , Rn) from each of the at least one GUI-component (220a,..., 220n). The control signal request (Ra, ... , Rn) defines a sub-set of the set of non-cursor controlling event output signals (D-HIi) which is required by the particular GUI-component (220a, .... 220n). The event engine (210) delivers non-cursor controlling event output signals (D-HIi) to the at least one GUI-component (220a,..., 220n) in accordance with each respective control signal request (Ra , Rn).

Full Text

THE BACKGROUND OF THE INVENTION AND PRIOR ART:
The present invention relates generally to computer based eye-tracking systems.
More particularly the invention relates to an arrangement for controlling a
computer apparatus associated with a graphical display and a corresponding
method which is being performed by a computer readable medium having a
program recorded therein.
The human computer interaction was revolutionized by the introduction of the
graphical user interface (GUI). Namely, thereby, an efficient means was provided
for presenting information to a user with a bandwidth which immensely
exceeded any prior channels. Over the years the speed at which information can
be presented has increased further through color screens, enlarged displays,
intelligent graphical objects (e.g. pop-up windows), window tabs, menus,
toolbars and sounds. During this time, however, the input devices have remained
essentially unchanged, i.e. the keyboard and the pointing device (e.g. mouse,
track ball or touch pad). In recent years, handwriting devices have been
introduced (e.g. in the form of a stylus or graphical pen). Nevertheless, while the
output bandwidth has multiplied several times, the input ditto has been
substantially unaltered. Consequently, a severe asymmetry in the communication
band-width in the human computer interaction has occurred.
In order to decrease this bandwidth gap, various attempts have

been made to use eye-tracking devices. However, in many
cases these devices miss the mark in one or several respects.
One problem is that the prior-art solutions fail to take a holistic
view on the input interfaces to the computer. Thereby, com-
paratively heavy motor tasks may be imposed on the eyes,
which in fact are strictly perceptive organs. Typically, this leads
to fatigue symptoms and a degree of discomfort experienced by
the user. This is particularly true if an eye tracker is used to
control a cursor on a graphical display, and for various reasons
the eye tracker fails to track the user's point of regard suffi-
ciently well, so that there is a mismatch between the user's
actual point of regard and the position against which the cursor
is controlled.
Instead of controlling the cursor directly, an eye gaze signal may
be used to select an appropriate initial cursor position. The do-
cument US, 6,204,828 discloses an integrated gaze / manual
cursor positioning system, which aids an operator to position a
cursor by integrating an eye-gaze signal and a manual input.
When a mechanical activation of an operator device is detected
the cursor is placed at an initial position which is predetermined
with respect to the operator's current gaze area. Thus, a user-
friendly cursor function is accomplished.
The document US, 6,401,050 describes a visual interaction sys-
tem for a shipboard watch station. Here, an eye-tracking camera
monitors an operator's visual scan, gaze location, dwell time,
blink rate and pupil size to determine whether additional cueing
of the operator should be made to direct the operator's attention
to an important object on the screen.
The document US, 5,649,061 discloses a device for estimating a
mental decision to select a visual cue from a viewer's eye
fixation and corresponding event evoked cerebral potential. An
eye tracker registers a viewing direction, and based thereon
fixation properties may be determined in terms of duration, start
and end pupil sizes, saccades and blinks. A corresponding

single event evoked cerebral potential is extracted, and an
artificial neural network estimates a selection interest in the
gaze point of regard. After training the artificial neural network,
the device may then be used to control a computer, such that
icons on a display are activated according to a user's estimated
intentions without requiring any manipulation by means of the
user's hands.
A few attempts have also been made to abstract user-generated
input data into high-level information for controlling a computer.
For example, the document US 2004/0001100 describes a
multimode user interface, where a flexible processing of a user
input is made possible without having to switch manually
between different input modes. Instead, within the data streams
different information categories are distinguished depending on
a context in which the data streams are generated.
Although this strategy may indeed enhance the efficiency of the
man-machine interaction, no multimodal solution has yet been
presented according to which eye-tracking data is processed
optimally. On the contrary, with only very few exceptions, to-
day's eye tracking interfaces are each tailored for one specific
task only. Thus, any processing of eye tracking data in respect
of a first application cannot be reused by a second application,
and vice versa. Hence, if multiple eye-controlled applications
are used in a single computer, one particular eye-tracking data
processing unit is typically required for each application. Natu-
rally, in such a case, there is a high risk that the different appli-
cations perform a substantial amount of overlapping eye-trac-
king data processing. Moreover, each designer of eye-control-
lable applications needs to have expertise in both eye tracking
technology and the interpretation of eye-tracking data in order to
extract the data required by the application.
SUMMARY OF THE INVENTION
The object of the present invention is therefore to provide a ho-

listic means of controlling a computer apparatus based on a
user's ocular activity, which alleviates the above problems and
thus offers an efficient man-machine interaction with a minimal
amount of double processing, i.e. wherein as much as possible
of the eye-tracking data processing is performed centrally in
respect of GUI-components that may belong to two or more se-
parate applications.
According to one aspect of the invention, the object is achieved
by the arrangement as initially described, wherein the event
engine is adapted to receive a control signal request from each
of the at least one GUI-component. The control signal request
defines a sub-set of the set of non-cursor controlling event
output signals, which is required by the GUI-component in
question. The event engine is also adapted to deliver non-cursor
controlling event output signals to the at least one GUI-compo-
nent in accordance with each respective control signal request.
This arrangement is advantageous because thereby a very
flexible interface is attained towards any applications which are
controllable by means of eye tracking signals. This, in turn,
enables software developers without an in-depth knowledge of
eye-tracking technology to design eye-controllable applications.
Therefore, the invention is believed to stimulate the develop-
ment of new such applications, and consequently render further
improvements of the human computer interaction possible.
Moreover, the computer's processing resources are freed for
alternative purposes, since a high-level eye-tracking data signal
derived in respect of one application may be reused by one or
more additional applications. In other words, the event engine is
adapted to perform a centralized eye-tracking data processing
for GUI-components (or potential "subscribers") on-demand.
Thus, each of these components and their requested control
signals need not be explicitly known at the point in time when
the event engine is implemented. Instead, according to the
invention, one or more GUI-components may request relevant
control signals in connection with the implementation of a later

added eye-controllable application. Of course, the opposite may
also be true, namely that a set of control signal requests is
presented once and for all at start-up of the proposed arrange-
ment.
According to a preferred embodiment of this aspect of the inven-
tion, the computer apparatus is also adapted to receive a cursor
control signal, and in response to the cursor control signal,
control a graphical pointer on the display. Thus, for instance, the
event engine and the cursor control signal may interact jointly
with GUI-components represented on the display, such that a
very intuitive man-machine interaction is accomplished.
According to another preferred embodiment of this aspect of the
invention, at least one GUI-component is adapted to generate at
least one respective output control signal upon a user mani-
pulation of the component. This means that by manipulating a
GUI-component, the user may cause the computer apparatus to
generate outgoing signals to one or more external units, such as
a printer, a camera etc. Naturally, this is a highly desirable
feature. Moreover, the fact that one or more GUI-components
may generate an output control signal does not preclude that
with respect to one or more other GUI-components, the non-
cursor controlling event output signals may exclusively affect the
component internally.
According to a preferred embodiment of this aspect of the inven-
tion, the event engine is adapted to produce at least a first
signal of the non-cursor controlling event output signals based
on a dynamic development of the eye-tracking data signal.
Thereby, a time parameter of the user's ocular activity may be
used to control functions and processes of the computer appa-
ratus. For instance, the time parameter may reflect a dwell time
of the user's gaze within a particular region on the display,
identify a certain gaze pattern etc. Many types of advanced eye-
controllable functions can thereby be realized.

According to yet another preferred embodiment of this aspect of
the invention, at least one GUI-component is adapted to
interpret a non-cursor controlling event output signal as a rep-
resentation of the user's intention. In response to this estimated
intention, a user manipulation of the component is triggered. For
example this may involve activating one or more computer
functions based on a command history. This is advantageous
because thereby the command input procedure may be simpli-
fied.
According to still another preferred embodiment of this aspect of
the invention, at least one GUI-component is adapted to
interpret a non-cursor controlling event output signal as an esti-
mated attention level of the user. In response to this estimated
attention level, a user manipulation of the component is
triggered. One advantage attainable thereby is that the com-
puter's behavior may be adapted to match a current attention
level of the user.
According to yet another preferred embodiment of this aspect of
the invention, at least one GUI-component is adapted to
interpret a non-cursor controlling event signal as a state-of-mind
parameter of the user, and in response thereto trigger a user
manipulation of the component. This feature is desirable
because it allows the computer to behave differently depending
on whether the user appears to be focused/concentrated,
distracted, tired/unfocused or confused etc.
According to still another preferred embodiment of this aspect of
the invention, the event engine is adapted to receive at least
one auxiliary input signal, such as a signal from a button or a
switch, a speech signal, a movement pattern of an input mem-
ber, a camera registered gesture pattern or facial expression, or
an EEG (electroencephalogram)-signal. On the further basis of
this auxiliary signal the event engine produces the set of non-
cursor controlling event output signals. Thereby, a highly
efficient combination of user input signals may be used to

control the computer. Generally, these combinations can be
made very intuitive and easy to learn.
According to another aspect of the invention the object is
achieved by the method as initially described, wherein a control
signal request is received from each of the at least one GUI-
component. The control signal request defines a sub-set of the
set of non-cursor controlling event output signals, which is
required by the particular GUI-component. Non-cursor con-
trolling event output signals are then delivered to the at least
one GUI-component in accordance with each respective control
signal request.
The advantages of this method, as well as the preferred embodi-
ments thereof, are apparent from the discussion hereinabove
with reference to the proposed arrangement.
According to a further aspect of the invention the object is
achieved by a computer program, which is directly loadable into
the internal memory of a computer, and includes software for
controlling the above proposed method when said program is
run on a computer.
According to another aspect of the invention the object is
achieved by a computer readable medium, having a program
recorded thereon, where the program is to control a computer to
perform the above proposed method.
The invention dramatically increases the available bandwidth for
transferring information from a user to a computer apparatus,
i.e. essentially generating commands, however not necessarily
perceived as such by the user. Therefore, this increase of the
bandwidth places no additional cognitive workload on the user.
On the contrary, by means of the invention, the cognitive
workload may, in fact, be reduced. At the same time, the
increased bandwidth vouches for an improved efficiency of the
man-machine interaction.

Moreover, by means of the invention, commands which traditio-
nally have required hand and/or finger manipulations, can be
efficiently and effortlessly effected based on the user's eye
activity. Naturally, this is desirable in a broad range of appli-
cations, from disabled computer users, support operators in a
call-center environment (e.g. when entering/editing data in a
customer relationship management application), users of
advanced computer aided design (CAD) tools, surgeons, to
drivers and pilots who, for various reasons, cannot effectively
produce hand- and/or finger-based commands. Even in
situations where the users have their hands and fingers readily
accessible, the invention may be useful to improve the ergono-
mics and the reduce risk of e.g. repetitive strain injuries.
Alternatively, the environment in which the computer apparatus
is placed may be so clean or dirty that either the environment
has to be protected from possible emissions from the computer
apparatus, or reverse this apparatus must be protected against
hazardous substances in the environment and therefore has to
be encapsulated to such a degree that a traditional entry of
input of commands is made impossible, or at least impractical.
The invention offers an excellent basis for developing new
software and computer applications, which are controllable by
the eyes of a user. Thereby, in the long term, the invention
vouches for a deep and unconstrained integration of eye inter-
action applications into the standard computer environment.
Further advantages, advantageous features and applications of
the present invention will be apparent from the following des-
cription and the dependent claims.
BRIEF DESCRIPTION OF THE DRAWINGS
The present invention is now to be explained more closely by
means - of preferred embodiments, which are disblosed as
examples, and with reference to the attached drawings.

DESCRIPTION OF PREFERRED EMBODIMENTS OF THE
INVENTION
Figure 1 shows an overview picture of a typical: use-case
according to the invention. Here, a user 110 controls a computer
apparatus 130, at least partly based on an eye-tracking data
signal DEYE, which describes the user's 110 point of regard x, y
on a display 120. Thus, by watching a representation of a GUI-
component 220 on the display 120, the user 110 may generate
commands to the computer apparatus 130. This manipulation is
enabled, since the GUI-component 220 is adapted to be, at leasl
indirectly, influenced by the eye-tracking data signal,DEYE. The
invention presumes that the eye-tracking data signal DEYE may
result in events, related to any task performable by the computer
apparatus, apart from affecting a cursor/pointer on the display
120. It should be noted that according to the invention, any type
of known computer screen or monitor, as well as combinations
of two or more separate displays may represent the display 120.
For example, the display 120 may constitute a pair of
stereoscopic screens, a heads-up display (HUD), a head-
mounted display (HMD) and a presentation means for a virtual
environment, such as the eyepieces of a pair of 3D-glasses or a
room where the walls include projection screens for presenting a
virtual environment.
Naturally, in order to produce the eye-tracking data signal DEYE,
the display 120 is associated with, or includes, an eye-tracker.
This unit is not a subject of the present patent application, and
therefore will not be described in detail here. However, the eye-
tracker is preferably embodied by the solution described in the
Swedish patent application 0203457-7, filed on 21 November
2002 in the name of the applicant.
Preferably, a graphics control signal C-GR is generated by the
computer apparatus 130 to accomplish visual feedback infor-
mation on the display 120. The visual feedback information is
generated in response to any user-commands received by the
computer apparatus 130, so as to confirm to the user 110 any

commands, which are based on the eye-tracking data signal
DEYE- Such a confirmation is especially desirable when the com-
mands are produced by means of perceptive organs; for exam-
ple the human eyes.
According to a preferred embodiment of the invention, besides
the eye-tracking data signal DEYE. the computer apparatus 130
is adapted to receive a cursor control signal K, which controls
the position of a graphical pointer on the display 120. Of course,
the graphics control signal C-GR may be based also on the
cursor control signal K.
Figure 2 shows an arrangement according to an embodiment of
the invention, which may be realized by means of the computer
apparatus 130 described above with reference to the figure 1.
The arrangement includes an event engine 210 and at least one
GUI-component 220, which is adapted to be manipulated based
on user-generated commands, at least partly expressed by the
eye-tracking data signal DEYE- The event engine 210 is adapted
to receive the eye-tracking data signal DEYE. and based thereon
produce a set of non-cursor controlling event output signals D-
Hlj that influence the at least one GUI-component 220. Each
non-cursor controlling event output signal D-Hlj, in turn, descri-
bes a particular aspect of the user's 110 ocular activity in
respect of the display 120.
For example, a first signal may indicate whether the user's 110
gaze is directed towards the display at all (i.e. a "gaze-on-
display" signal), a second non-cursor controlling event output
signal may reflect a dwell time of the user's 110 gaze within a-
certain area on the display 120, a third signal may designate a
gaze fixation (at a specific point), a fourth signal may indicate
whether the gaze saccades, a fifth signal may indicate whether
the gaze follows a smooth path, a sixth signal may reflect that
the user 110 reads a text, and a seventh signal may be triggered
if the user 110 appears to be distracted, based on the particular

eye-tracking data signal DEYE that he/she produces.
According to the invention, the event engine 210 receives a
respective control signal request Ra, ..., Rn from each of the at
least one GUI-component 220a, ..., 220n. The control signal
request, say Ra, defines a sub-set of the set of non-cursor con-
trolling event output signals D-Hli which are required by the par-
ticular GUI-component, say 220a, to operate as intended. The
event engine 210 then delivers non-cursor controlling event
output signals D-Hli to each of the at least one GUI-component
220a, ..., 220n in accordance with the respective control signal
request Ra, ..., Rn.
A most efficient processing is accomplished if the event engine
210 exclusively produces those event output signals D-Hli which
are actually requested by at least one GUI-component.
However, according to the invention, it is also conceivable that
all non-cursor controlling event output signals D-Hli that are
possible to produce are always generated by the event engine
210, irrespective of whether a corresponding control signal
request has been received or not. Namely, this simplifies the
procedure, and depending on the application, this strategy may
not require an overly extensive processing.
According to a preferred embodiment of the invention, each
GUI-component 220a, .... 220n is adapted to generate at least
one respective output control signal Ca, ..., Ca upon a user
manipulation of the component 220a, ..., 220n. Thus, in addition
to generating the above-mentioned visual feedback information,
one or more internal or peripheral devices may be influenced by
means of the output control signals Ca, ..., Ca. For example a
print job may be initiated, a computing task may be executed, an
e-mail may be sent out, a camera may be triggered to take a
picture etc.
As mentioned above, the non-cursor controlling event output
signals D-HIi may describe many different aspects of the eye-

tracking data signal DEYE- According to one embodiment of the
invention, at least one output signal D-Hli is based on a dynamic
development of the eye-tracking data signal DEYE- Thereby, the
signal can represent a particular gaze pattern over the display
120. The gaze pattern, in turn, may be determined, for instance
to constitute saccades, a smooth pursuit, periods of fixation or
reading.
A non-cursor controlling event output signal D-Hli may also
indicate gaze-enter/gaze-leave data. This data is a parameter
which reflects the time instances when the eye-tracking data
signal DEYE indicates that the user's point of regard falls within a
GUI component's representation on the display. Hence, the
gaze-enter data is generated when the user's gaze falls onto a
GUI component's representation on the display, and the gaze-
leave data is generated when the gaze is directed outside this
representation.
The above-mentioned dwell time for a GUI-component is typi-
cally defined as the period between the gaze-enter data and the
gaze-leave data with respect to a particular GUI component. It is
normally preferable to link an activation signal to the dwell time,
such that for instance an "eye-button" is activated when a
certain dwell time is reached. Alternatively, a button, a switch, a
speech signal, a movement pattern of an input member, a
camera registered gesture pattern or facial expression may
constitute the activation signal. Moreover, an eye blink or a pre-
defined EEG-signal may cause the activation signal. However,
the latter types of signals are usually relatively difficult for the
user to control with a sufficient accuracy.
According to one embodiment of the invention, the event engine
210 is adapted to receive at least one auxiliary input signal DJ,
and produce the set of non-cursor controlling event output
signals D-Hli on the further basis of this signal. The auxiliary
input signal DJ may originate from a button, a switch, a speech
signal, a movement pattern of an input member, a camera

registered gesture pattern or a facial expression, or an EEG-
signal.
Thereby, based on combinations of the eye-tracking data signal
DEYE and one or more auxiliary input signals DJ composite user
commands may be created, which are very intuitive and easy to
learn. Hence, a highly efficient man-machine interaction with the
computer apparatus 130 may be accomplished. For instance,
watching an eye-button for the document and uttering the
control word "open" can open a text document. If a more
complex speech recognition is available, an Internet search may
be effected by focusing the gaze towards a relevant text input
box while uttering the desired search terms, and so on.
According to one embodiment of the invention, at least one GUI-
component 220a, ..., 220n is adapted to'interpret a non-cursor
controlling event output signal D-Hli from the event engine 210
as an estimated intention of the user 110. Then, in response to
the estimated intention, a user manipulation of the component
220a, ..., 220n is triggered. Preferably, the event engine 210
estimates a user intention based on multiple input sources
received as auxiliary input signals DJ, which may include a key-
board signal, a mouse signal, voice data and camera images.
However, naturally, the eye-tracking data signal DEYE may also
constitute a basis for the estimated user intention. For example,
important information can be drawn from different gaze patterns
and fixation times.
According to another embodiment of the invention, at least one
GUI-component 220a, ..., 220n is adapted to interpret a non-
cursor controlling event output signal D-Hli from the event
engine 210 as an estimated attention level of the user 110.
Correspondingly, a user manipulation of the component 220a,
..., 220n is triggered in response to the estimated attention
level. Also the attention level may be estimated based on the
auxiliary input signals DJ, for instance originating from the
keyboard, the mouse, voice data and camera images, and the

eye-tracking data signal DEYE- Particularly, gaze patterns,
fixation points and fixation times constitute an important basis
for determining the user's 110 attention level. Preferably, the
GUI-components 220a, ..., 220n vary their behavior depending
on the estimated attention level, such that the components'
characteristics match the user's 110 current performance.
Furthermore, according to one embodiment of the invention, at
least one GUI-component 220a, .... 220n is adapted to interpret
a non-cursor controlling event output signal D-Hli from the event
engine 210 as a state-of-mind parameter of the user 110. The
state-of-mind parameter reflects a general user 110 status, for
example whether he/she appears to be focused/concentrated,
distracted, tired/unfocused or confused. For example, the state-
of-mind parameter may indicate an approximated 20% degree of
tiredness and an approximated 50% degree of attention. Then,
based on the estimated state-of-mind, a user manipulation of the
component 220a, ..., 220n is triggered. Typically, the number of
and the contents of any help menus and pop-up windows may
be adapted in response to the estimated state-of-mind. How-
ever, in operator environments, such as radar watch stations where the attention level as well as the state-of-mind may be
truly critical, the security may be improved by pointing out
targets etc. that the operator has not yet observed.
According to one embodiment of the invention, the event engine
210 is associated with a template library 230, which contains
generic GUI-components, such as eye-buttons, scroll bars,
multiview toolbars (see below with reference to figure 4), text
input fields (see below with reference to figure 5), expandable
text input fields (see below with reference to figure 6) and scroll
windows (see below with reference to figure 7). Thereby, by
means of the template library 230, a software designer may
conveniently create functions and controls which can be
manipulated based on a user's ocular activity in respect of a
display. Of course, after completing the design of a particular
eye-controllable application, however, the template library 230

has no actual function in respect of the GUI-components that
may originate from its generic components. Nevertheless, the
template library 230 may again become useful in case of a
future upgrade or a redesign of the application.
According to the invention, it is not technically necessary that
any GUI-component manipulations be graphically confirmed on
the display. However, this is generally preferable from a user-
friendliness point-of-view. Figure 3a shows a schematic symbol
310, which represents an eye-controllable GUI-component on a
display that is set in a non-observed mode (i.e. the eye-tracking
data signal DEYE indicates that the user's point of regard lies
outside the symbol 310). Figure 3b shows the symbol 310 in an
observed mode, which is, set when the eye-tracking data signal
DEYE indicates that the user's point of regard falls within the
display area represented by the symbol 310. In the observed
mode, the symbol 310 contains a centrally located object 311.
This object 311 confirms to the user that the computer
apparatus has registered that his/her gaze presently is directed
towards the symbol 310. Thus, any manipulations in respect of
the GUI-component associated with the symbol 310 can be
performed. An important advantage attained by means of the
centrally located object 311 is that this object assists the user in
focusing his/her gaze to the center of the symbol 310. Thereby,
a more reliable eye-tracking function is accomplished, and for a
given eye-tracker, the symbols 310 can be made smaller than
otherwise. Of course, the symbol 310 and the centrally located
object 311 may have any other outline than the square repre-
sentation of figures 3a and 3b. Moreover, to further improve the
visual cueing, the object 311 may be animated and or have a
particularly interesting color or shape.
Figure 4 illustrates a first embodiment according to the inven-
tion, where a proposed multiview toolbar 401 is used to control
applications in a frame 400. The multiview toolbar 401 here
includes four different eye-buttons, which each may contain a
thumbnail image (not shown) of a respective application with

instance as shown in the figure 6b. Hence, more information
than what was initially visible in the field 620 may be shown.
The thus expanded text field 620 may even cover graphical
objects which otherwise are shown in the frame 400. This is ad-
vantageous, because thereby the information may be presented
on-demand, so that the frame 400 and the sub-frame 520
contain more data than what actually can be fitted therein. For
instance, a text scrolling, which otherwise would have been
necessary can be avoided.
Figure 7 illustrates a fourth embodiment according to the
invention, which realizes a scrolling function based on a user's
ocular activity. Here, a third application, e.g. a map viewer, is
associated with a third button 710. Thus, by watching the third
button 710, and activating its associated GUI-component, either
based on a gaze dwell time or by means of a separate activation
signal, the computer apparatus opens up a map sub-frame 720
within the frame 400. This sub-frame, in turn, presents a digi-
tized map. It is here presumed that the map is larger than the
available presentation area in the map sub-frame 720, so that
only a part of the map can be presented at the same time.
A scrolling with- respect to the map is achieved based on the
user's point or regard. Preferably, no scrolling occurs as long as
the eye-tracking data signal indicates that the point of regard
lies within a central area delimited by a first dashed line a, a
second dashed line b, a third dashed line c and a fourth dashed
line d, neither of which preferably are visible on the display. If
however, the user's point of regard is placed outside any of the
lines a, b, c or d, the map scrolls in a particular direction given
by the point of regard. Specifically, this means that a point of
regard below the line c results in a downward scroll along the
arrow S; a point of regard above the line d results in a upward
scroll along the arrow N; a point of regard to the right of the line
b results in a rightward scroll along the arrow E; and a point of
regard to the left of the line a results in a leftward scroll along
the arrow W. Moreover, a point of regard, which lies below the

line c and to the right of the line b results in a diagonal scroll
along the arrow SE; a point of regard, which lies above the line
d and to the right of the line b results in a diagonal scroll along
the arrow NE; a point of regard, which lies above the line d and
to the left of the line a results in a diagonal scroll along the
arrow NW; and a point of regard, which lies below the line c and
to the left of the line a results in a diagonal scroll along the
arrow SW. This scroll function may either be activated based on
a dwell time, or a separate activation signal may be required,
such as a clicking a key/button or holding down a key/button.
Furthermore, the scroll speed may depend on the distance
between the point of regard and the respective lines a, b, c and
d, such that a relatively long distance corresponds to a compa-
ratively high speed, and vice versa. The scroll speed may also
depend on the scroll time, and/or a length of the latest saccades
registered in the eye-tracking data signal. Preferably, a maxi-
mum scroll speed is set to such a value that the scrolled infor-
mation is visible to the user at all possible speeds.
According to one embodiment of the invention, the scroll
function is stopped by pressing a key/button, releasing a
key/button, the length of the latest saccades exceeding a
particular value, the point of regard moving outside the map
sub-frame 720, the point of regard moving towards the center of,
the map sub-frame 720, or towards an opposite scroll activation
line a, b, c, or d.
It should be noted that, eyetrack-driven scrolling solutions as
such are described in the prior art, for instance in the document
US, 5,850,221. Here, a-page oriented or continuous information
scrolling function is initiated or controlled based on where a
viewer's eyes are looking.
To sum up, the general method of controlling a computer appa-
ratus according to the invention will now be described with
reference to the flow diagram in figure 8.

An initial step 810 receives a control signal request from each of
at least one GUI-component, which is adapted to be manipu-
lated based on user-generated, eye commands. The control
signal request defines a sub-set of the set of non-cursor con-
trolling event output signals, which is required by the particular
GUI-component in order to operate as intended.
A step 820 then receives an eye-tracking data signal, which des-
cribes a user's point of regard on a display associated with a
computer that embodies the at least one GUI-component.
Subsequently, a step 830 produces a set of non-cursor con-
trolling event output signals based on the eye-tracking data
signal, plus any auxiliary input signal.
Subsequently, a step 840 delivers the non-cursor controlling
event output signals to the at least one GUI-component so that a
relevant influence thereof may occur (i.e. according to each
respective control signal request). It is presumed that each non-
cursor controlling event output signal describes a particular
aspect of the user's ocular activity in respect of the display.
Thus, the non-cursor controlling event output signals express
user-generated eye commands. After the step .840, the pro-
cedure loops back to the step 820 for receiving an updated eye
tracking data signal.
All of the process steps, as well as any sub-sequence of steps,
described with reference to the figure 8 above may be controlled
by means of a programmed computer apparatus. Moreover,
although the embodiments of the invention described above with
reference to the drawings comprise computer apparatus and
processes performed in computer apparatus, the invention thus
also extends to computer programs, particularly computer
programs on or in a carrier, adapted for putting the invention
into practice. The program may be in the form of source code,
object code, a code intermediate source and object code such
as in partially compiled form, or in any other form suitable for
use in the implementation of the process according to the

invention. The program may either be a part of an operating
system, or be a separate application. The carrier may be any
entity or device capable of carrying the program. For example,
the carrier may comprise a storage medium, such as a Flash
memory, a ROM (Read Only Memory), for example a CD
(Compact Disc) or a semiconductor ROM, an EPROM (Erasable
Programmable Read-Only Memory), an EEPROM (Electrically
Erasable Programmable Read-Only Memory), or a magnetic recording medium, for example a floppy disc or hard disc.
Further, the carrier may be a transmissible carrier such as an
electrical or optical signal which may be conveyed via electrical
or optical cable or by radio or by other means. When the
program is embodied in a signal which may be conveyed directly
by a cable or other device or means, the carrier may be
constituted by such cable or device or means. Alternatively, the
carrier may be an integrated circuit in which the program is
embedded, the integrated circuit being adapted for performing,
or for use in the performance of, the relevant processes.
The term "comprises/comprising" when used in this specification
is taken to specify the presence of stated features, integers,
steps or components. However, the term does not preclude the
presence or addition of one or more additional features,
integers, steps or components or groups thereof.
The invention is not restricted to the described embodiments in
the figures, but may be varied freely within the scope of the
claims.

We Claim:
1. An arrangement for controlling a computer apparatus (130) associated
with a graphical display (120), the display (120) presenting a representation
of at least one GUI-component (220) which is adapted to be manipulated
based on user-generated commands, and at least one of the at least one
GUI-component (220a,... 220n) is adapted to generate at least one respective
output control signal (Ca ...Ca) upon a user manipulation of the component
(220a,... 220n), the arrangement comprising an event engine (210) adapted
to receive an eye-tracking data signal (DEYE) describing a user's (110) point of
regard (x, y) on the display (120), and at least based on the eye-tracking
data signal (DEYE) produce a set of non-cursor controlling event output signals
(D-HIi) influencing the at least one GUI-component (220), each non-cursor
controlling event output signal (D-HIi) describing a particular aspect of the
user's (110) ocular activity in respect of the display (120), characterized in
that the event engine (210) is adapted to:
receive a control signal request (Ra, ..., Rn) from each of the at least one GUI-
component (220a, ..., 220n), the control signal request (Ra, ..., Rn) defining a
sub-set of the set of non-cursor controlling event output signals (D-HIi) which
is required by the GUI-component (220a, ..., 220n) to operate as intended to

produce the event output signals (D-HIi) which are requested by the at least
one GUI-component (220a,...,220n) in the control signal request (Ra,...,Rn),
and
deliver non-cursor controlling event output signals (D-HIi) to the at least one
GUI-component (220a, ..., 220n) in accordance with each respective control
signal request (Ra, ..., Rn).
2. An arrangement as claimed in claim 1, wherein the computer apparatus
(130) is adapted to:
receive a cursor control signal (K), and
control a graphical pointer on the display (120) in response to the cursor
control signal (K).
3. An arrangement as claimed in any one of the preceding claims, wherein
the event engine (210) is adapted to produce at least a first signal of the
non-cursor controlling event output signals (D-HIi) based on a dynamic
development of the eye-tracking data signal (DEYE).
4. An arrangement as claimed in claim 3, wherein the first signal represents
a particular gaze pattern over the display (120).

5. An arrangement as claimed in claim 4, wherein at least one GUI-
component (220a, ..., 220n) is adapted to interpret the first signal as an
estimated intention of the user (110), and trigger a user manipulation of
the component (220a,..., 220n) in response to the estimated intention.
6. An arrangement as claimed in any one of the claims 4 or 5, wherein at
least one GUI-component (220a, ..., 220n) is adapted to interpret the first
signal as an estimated attention level of the user (110), and trigger a user
manipulation of the component (220a, ..., 220n) in response to the
estimated attention level.
7. An arrangement as claimed in any one of the claims 4 to 6, wherein at
least one GUI-component (220a, ..., 220n) is adapted to interpret the first
signal as a state-of-mind parameter of the user (110), and trigger a user
manipulation of the component (220a, ..., 220n) in response to the state-
of-mind parameter.
8. An arrangement as claimed in any one of the preceding claims, wherein
the event engine (210) is adapted to receive at least one auxiliary input
signal (DJ) and produce the set of non-cursor controlling event output
signals (D-HIi) on the further basis of the at least one auxiliary input
signal (DJ).

9. An arrangement as claimed in any one of the preceding claims, wherein
the at least one auxiliary input signal (DJ) is originated from at least one
of a button, a switch, a speech signal, a movement pattern of an input
member, a gesture pattern, a facial expression and an EEG-signal.
10. A method of controlling a computer apparatus (130) associated with
graphical display (120), the display (120) representing at least one GUI-
component (220) which is adapted to be manipulated based on user-
generated commands, the method comprising:
receiving an eye-tracking data signal (DEYE) which describes a user's (110)
point of regard (x, y) on the display (120), and
producing a set of non-cursor controlling event output signals (D-HIi) at least
based on the eye-tracking data signal (DEYE), the set of non-cursor controlling
event output signals (D-HIi) influencing the at least one GUI-component
(220), and each non-cursor controlling event output signal (D-HIi) describing
a particular aspect of the user's (110) ocular activity in respect of the display
(120), the method characterized by
receiving a control signal request (Ra, ... , Rn) from each of the at least one
GUI-component (220a, ..., 220n), the control signal request (Ra, ..., Rn)
defining a sub-set of the set of non-cursor controlling event output signals
(D-HIi) which is required by the GUI-component (220a, ..., 220n) to operate
as intended

producing the event output signals (D-HIi) which are requested by the at
least one GUI-component (220a,...,220n) in the control signal request
(Ra,...,Rn), and
delivering non-cursor controlling event output signals (D-HIi) to the at least
one GUI-component (220a, ..., 220n) in accordance with each respective
control signal request (Ra, ..., Rn).
11. A method as claimed in claim 10, comprising:
receiving a cursor control signal (K), and
controlling a graphical pointer on the display (120) in response to the cursor
control signal (K).
12. A method as claimed in any one of the claims 10 or 11, comprising
producing at least a first signal of the non-cursor controlling event output
signals (D-HIi) based on a dynamic development of the eye-tracking data
signal (DEYE).
13. A method as claimed in claim 12, comprising the first signal representing a
particular gaze pattern over the display (120).
14. A method as claimed in claim 13, comprising at least one GUI-component
(220a, ..., 220n) being adapted to interpret the first signal as an
estimated intention of the user (110), and trigger a user manipulation of
the component (220a,..., 220n) in response to the estimated intention.

15. A method as claimed in any one of the claims 13 or 14, comprising at
least one GUI-component (220a, ..., 220n) being adapted to interpret the
first signal as an estimated attention level of the user (110), and trigger a
user manipulation of the component (220a, ..., 220n) in response to the
estimated attention level.
16. A method as claimed in any one of the claims 13 to 15,comprising at least
one GUI-component (220a, ... , 220n) being adapted to interpret the first
signal as a state-of-mind parameter of the user (110), and trigger a user
manipulation of the component (220a, ..., 220n) in response to the state-
of-mind parameter.
17. A method as claimed in any one of the claims 11 to 18,
comprising:
receiving at least one auxiliary input signal (DJ), and
producing the set of non-cursor controlling event output signals (D-HIi) on
the further basis of the at least one auxiliary input signal (DJ).

18. A method as claimed in any one of the claims 10 to 17,comprising the at
least one auxiliary input signal (DJ) being originated from at least one of a
button, a switch, a speech signal, a movement pattern of an input
member, a gesture pattern, a facial expression and an EEG-signal.

(57) Abstract: The present invention relates to a computer based eye-tracking solution. A computer apparatus is associated with
a graphical display, which in turn, represents at least one GUI-component (220a 220n) that is adapted to be manipulated based
on user-generated commands. An event engine (210) is adapted to receive an eye-tracking data signal (DEYE) that describes a user's
point of regard on the display. Based on the signal (DEYE), the event engine (210) is adapted to produce a set of non-cursor controlling
event output signals (D-HIi), which influence the at least one GUI-component (220a, ..., 220n). Each non-cursor controlling event
output signal (D-HIi) describes a particular aspect of the user's ocular activity in respect of the display. Initially, the proposed event
engine (210) receives a control signal request (Ra,... , Rn) from each of the at least one GUI-component (220a,..., 220n). The control
signal request (Ra, ... , Rn) defines a sub-set of the set of non-cursor controlling event output signals (D-HIi) which is required by the
particular GUI-component (220a, .... 220n). The event engine (210) delivers non-cursor controlling event output signals (D-HIi) to
the at least one GUI-component (220a,..., 220n) in accordance with each respective control signal request (Ra , Rn).

Documents:

03422-kolnp-2006 abstract.pdf

03422-kolnp-2006 claims.pdf

03422-kolnp-2006 correspondence others.pdf

03422-kolnp-2006 description(complete).pdf

03422-kolnp-2006 drawings.pdf

03422-kolnp-2006 form-1.pdf

03422-kolnp-2006 form-2.pdf

03422-kolnp-2006 form-3.pdf

03422-kolnp-2006 form-5.pdf

03422-kolnp-2006 international publication.pdf

03422-kolnp-2006 international search authority report.pdf

03422-kolnp-2006-correspondence others-1.1.pdf

03422-kolnp-2006-correspondence-1.2.pdf

03422-kolnp-2006-form-26.pdf

03422-kolnp-2006-international search authority report-1.1.pdf

03422-kolnp-2006-pct request.pdf

3422-KOLNP-2006-(07-06-2013)-ANNEXURE TO FORM 3.pdf

3422-KOLNP-2006-(07-06-2013)-CLAIMS.pdf

3422-KOLNP-2006-(07-06-2013)-CORRESPONDENCE.pdf

3422-KOLNP-2006-(07-06-2013)-DESCRIPTION (COMPLETE).pdf

3422-KOLNP-2006-(07-06-2013)-FORM-1.pdf

3422-KOLNP-2006-(07-06-2013)-FORM-2.pdf

3422-KOLNP-2006-(07-06-2013)-FORM-5.pdf

3422-KOLNP-2006-(07-06-2013)-OTHERS.pdf

3422-KOLNP-2006-(07-06-2013)-PETITION UNDER RULE 137-1.pdf

3422-KOLNP-2006-(07-06-2013)-PETITION UNDER RULE 137.pdf

3422-KOLNP-2006-(27-01-2014)-CLAIMS.pdf

3422-KOLNP-2006-(27-01-2014)-CORRESPONDENCE.pdf

3422-kolnp-2006-CANCELLED PAGES.pdf

3422-KOLNP-2006-CORRESPONDENCE.1.1.pdf

3422-kolnp-2006-CORRESPONDENCE.pdf

3422-kolnp-2006-DECISION.pdf

3422-kolnp-2006-EXAMINATION REPORT.pdf

3422-kolnp-2006-form 18.pdf

3422-kolnp-2006-FORM 26.pdf

3422-kolnp-2006-GRANTED-ABSTRACT.pdf

3422-kolnp-2006-GRANTED-CLAIMS.pdf

3422-kolnp-2006-GRANTED-DESCRIPTION (COMPLETE).pdf

3422-kolnp-2006-GRANTED-DRAWINGS.pdf

3422-kolnp-2006-GRANTED-FORM 1.pdf

3422-kolnp-2006-GRANTED-FORM 2.pdf

3422-kolnp-2006-GRANTED-FORM 3.pdf

3422-kolnp-2006-GRANTED-FORM 5.pdf

3422-kolnp-2006-GRANTED-SPECIFICATION-COMPLETE.pdf

3422-kolnp-2006-INTERNATIONAL PUBLICATION.pdf

3422-kolnp-2006-OTHERS.pdf

3422-kolnp-2006-REPLY TO EXAMINATION REPORT.pdf

abstract-03422-kolnp-2006.jpg

« Previous Patent

Next Patent »

Patent Number

263194

Indian Patent Application Number

3422/KOLNP/2006

PG Journal Number

42/2014

Publication Date

17-Oct-2014

Grant Date

13-Oct-2014

Date of Filing

20-Nov-2006

Name of Patentee

TOBII TECHNOLOGY AB

Applicant Address

KARLSROVAGEN 2D,SE-18253 DANDERYD.

Inventors:

#	Inventor's Name	Inventor's Address
1	BJORKLUND,CHRISTOFFER	ANGSKARSGATAN 3,SE-11529 STOCKHOLM.
2	JACOBSON,MAGNUS	FREDSGATAN 15,SE-17233 SUNDBYBERG.
3	SKOGO,MARTEN	SVARTVIKSSLINGAN 25,SE-16738 BROMMA
4	ESKILSSON,HENRIK	KVARNVAGEN 53,SE-18239 DANDERYD.

PCT International Classification Number

G06F3/00; G06F3/00

PCT International Application Number

PCT/SE2005/000775

PCT International Filing date

2005-05-24

PCT Conventions:

#	PCT Application Number	Date of Convention	Priority Country
1	04445071.6	2004-06-18	EUROPEAN UNION