Title of Invention

"THREE DIMENSIONAL DIGITAL ENTITY MAGNIFYING GLASS SYSTEM INCORPORATED THREE DIMENSIONAL VISUAL TRAINING FUNCTIONS."

Abstract The present invention provides a three-dimensional digital entity magnifying glass technique-assisting and training/educational distribution system incorporating three-dimensional visual training functions by means of image composition that enables a. three-dimensional visual instruction containing a depth when giving visual instructions to an HMD worn by a medical practitioner, Solution for Problem A three-dimensional digital entity magnifying glass technique-assisting and training/educational distribution system incorporating three-dimensional visual training functions by means of image composition for utilizing digital entity magnifying glasses of identical standards worn by a medical practitioner and an instructor to thereby display only images of a pointing device or various kinas of instruments among image information of the instructor's CCD camera and synchronously and compositely displaying a three-dimensional visual training image output from an image-processing apparatus that can display an after-image of the image for an arbitrary time setting into image information of the CCD camera of the digital entity magnifying glass worn by the medical practitioner, thereby providing equal-scaling display of image information to which an instruction and a comment by use of a visual display/instruction image are added three-dimensionally as well as an instrument actually used by the instructors (groups of lecturers) on the three-dimensional HMD of the medical practitioner. Selected drawing: FIG. 2
Full Text DESCRIPTION

T h r e e - D i m e n s i o i .-\ L u S y s t e m I n c o r p o r a t i n g I i i r e c F u n c t i o n s

Technical Field
T he present i n \ .- n t i. f hi techniques by whic h a 3 u r c: operation is performer ny a a s u r g i c a 1 ope r a t i o n • :. o t through visual i n s t r u
i relates t. o a system r o r , y by utilizing a communication v. . More specifically, the e in for distributing a highly rechnical educational, program i o p e r a t i o n or any o t h e r m-a n u a 1 direct observation, performing hly precision oriented c a 1 operation o r: any o c h e r pecific customer (that, is, such .? r operations is controlled or oral instructions by use of •position based on information d of a person with no technical ;:.'Ot from a remote location) , or o f an invisible region together d image to a visual perception , :> e ration without a need for the ; i i G eyes off the surgical field.

Back g r o and A r t
Generally, in r.he c se of providing training
regarding a surgical nper technique in the c .1 i.n^.-al old, a medical practitioner and an instructor face d'-fez nt visual field directions and adopt different poatU'.-.-s, ... .d it is thus difficult to p.rovide training regarding manual technique such as accurate in s t r nuient.a t i on , whi *
practitioner's view i t! ••.• the opera tiilri may have to be
interrupted t emporar •. L y; urther, it may also cause a
patient to reel uneasv in ; conscious operating environment
and suffer' from a ps / :hot; nic shock.
Conversely, in ".he ise of instructing a student to
perfij in. a highly pic •:. :ii- oriented technique .such as a
surgical operation b ; an nstructor, if .the technique and
the surgical operatic• i on > closed site are to be observed
from identical ideal -/.I.sua field as the instructing medical
p r a c t .1 t i oner ( .1 e c t u r • •- r ) , .. order t o p e r rn i t a p 1 u r a .1. i t y of
students (customers) •_ o :: .uentially observe an operating
situation at each step of he surgical operation, not only
extremely much of tine if required but also the number of
persons who can obsc r ve • •••- operation simultaneously is
limited by the physical c istraints of the facilities used
for the operation, a.: i su i lectures have been observed by

very few people. In ...Idii on, if the instructing medical practitioner (Lecture-M a: i the student (customer) speak different languages, . n i: •. Moreover, both in t :; case of instructing a medical practitioner and the :::ase of an instructor instructing a student, an operator ieed to once take his eyes off a surgical field and c jsel watch and confirm information such as an X-ray picture, , CT picture, an MRI picture, or an electrocardiogram, so t.. at it has been difficult to share the information with orhei : especially at a remote location in real time.
Further, in the ca.v; • of a surgical operation, it is difficult to transport a pa lent himself to a remote location for an educational piogra.i; also, in the case of other special techniques ];...-. rfoi: :ed by hand which require large facilities or unmovabJe in.- uruments, a visiting student used to arrange his schedule s as to fit in with the operating date and spend much t rans; ortation costs and time required by transportation. Also, in the case of distributing an educational program t: o a emote location by using -various types of communicati Ki me..us, if a surgical operation on an individual is to be d:i ,-clo: :;d to a third party fundamentally, to obtain consent of a patient himself for responsibility
of privacy protection, of -ourse, the much the same responsibility is imposed ( ;i information so prevent it from being leaked along a commun cation path. Conversely, in the case of training on i.cal operation or a practical training of highly precis1on oriented operation which is performed at a Remote loc>: cioh; instructors (groups of lecturers) must go to a sp I that requires a lot of costs, so that various costs (tr, importation costs and time required by transportatioi) related to the training and, moreover, the remote area -equiring much time to go there could not easily acC'Mumod i.e performing of an emergency surgical operation; e spec.; .illy in the case of training by a plurality of instructors (groups of lecturers), the costs
involved have swollen even urther and an emergency surgical
*• operation has proved even more difficult to conduct.
Furthermore, it; is rohibited by medical licensing legal regulations to condi ct medical operation abroad, so that it is basically impo.-,::-»ible to conduct a lecture of a live operation visit i.ng educational program and practical training of a surgical op. ration by which the instructors (groups of lecturers) themselves perform the surgical operation abroad.
Further, if a medic tl practitioner (customer) actually wishes to be trained in a surgical operation or any other highly precision 01 ented techniques conducted manually, it is essentially impossible to gather instructors
(groups of lecturers) and persons with excellent skills and
knowledge who are scattered around to one place at the same
i-date and time so that a surgical operation may be edited or
managed. Furthermore, there has been no system for discussing a shared three iimensional image while displaying a three-dimensional instruction image.
Patent Document 1 Japanese Patent Application No. 2556829 Disclosure of the Invention
However, as described above, in the -case of instructing a surgical opt-ration or any other specialized manual techniques at a c,l riical field, generally an instructor would share identical visual field as a medical practitioner to conduct accurate training, thereby preventing required .-.-surge-, y time from increasing. Furthermore, even in the ^ase of conscious operating environments, what should be kept in mind is to avoid a-patient from being worrie-; about the existence of the instructor so that ho may not feel uneasy unnecessarily.
In contrast, in tht case of training students on highly precision oriented techniques such as a surgical operation which is conduc:ed by an instructor, if the students are permitted to watch the techniques and the surgical operation conduc ed on a closed site of the body from identical ideal visua field as the instructing medical practitioners (lecturers) many people are enabled to view
the field. In addition, i the instructing medical practitioner (lecturer) a:.d the student (customer) speak different languages, a ne-.-d is eliminated for causing an interpreter to be present curing the surgical procedure all the way, thereby avoiding .in increase in a fee per student (customer) for participating in such an educational program.
Moreover, also in the case of training a medical practitioner and the case of an instructor training a student, an operator is e cabled to view and confirm the information such as an X-j ;iy picture, a CT picture, an MRI picture, and an elect.roca diogram without taking his eyes off the surgical field an Further, in the ca o of a surgical operation, a patient need not go to a i .uaote location to participate in an educational program, t > instead reduce transportation fees of a visiting student (customer) and time required by the transportation. Alsc in the case of distributing the program to a remote locat. on by using the various types of communication means, information about individuals are prevented from being leaded. Conversely, in the case of conducting practical trailing on highly precision oriented operation or training on a surgical operation at a remote location, it is made pos. ible to reduce various costs (transportation costs am. time required by the transportation) of the instructors (group of lecturers) and
accommodate performing of an emergency surgical operation at a remote location which requires much of time to go there and, additionally, conduct: training by a plurality of instructors (groups of lecturers). Besides, it is made possible to avoid medical licensing legal regulations and conduct a lecture of a 1 iv operation visiting educational program and practical tra.'iing of a surgical operation by which the instructors (g i >ups of lecturers) themselves p e r f o i: rn t h c-' s u r g i Further/ if a in edit i I. practitioner (cu stonier) actually wishes to be trail e d in a surgical operation or any other highly precision 01 . anted, techniques c o n d u. c t e d manually, it is made possi Le to gather instructors (groups of lecturers) and person;.- v.; i t h excellent skills and knowledge who are scatter-.. , around to one place at the same date and time so that a su jical operation may be edited or managed and, furthermore, discuss their shared three-dimensional images in real time.
The present invention provides a system that provides a three-dimensional -digit,;! entity magnifying glass with a variety of functions, foj performing an assistance and training/educational prot-'am for surgical operations and other highly precision o '•• Lented techniques which are performed under direct obs- rvation of a human being, by using the internet or an intranet as communication means, the system incorporating a three-dimensional visual training
function for using an app,;tatus comprised of three-dimensional telescopic digital .magnifying glass which are constituted of a pair as to be worn by a medical practitioner and a instruct jr when performing the assistance and training/educational program for surgical operations and other highly precision oriented techniques and three-dimensional telescopic digital magnifying glass constituted of an automat c focusing CCD camera equipped with a pair of zoom lenses i.hich are set from a position that corresponds to a center between the right and left,pupils of workers wearing the three-dimensional digital entity magnifying glass cori:> t i t u . .-'d of the LCDs in front thereof in a direction toward a v sua 1 field of the naked eye, to extract only a three-dimensional image of a three-dimensional pointing device or various types of instruments (dummies) identical to instruments used, by the medical practitioner that • .m be helu and freely rnove-d within the visual field of the instructor and is operated by the instructor for g i v i n cj v i s u -i 1 instructions as recognizing on aii LCD monitor of a three-dimensional digital entity magnifying glass apparatu.. in front of a background having a single color of red, gr-,en, and blue (red is preferable in the mouse cavity becau; ^ it is complementary to a color of the blood vessel and t .te fresh) in consideration of a
complementary color rrelat onship corresponding to a background color of a visual, field image among the R, G, and B colors which are set; 30 a • to cover an entire visual field in front of the CCD camera , f the three-dimensional digital entity magnifying glass a paratus worn by the instructor captured by the CCD camera of the three-dimensional digital entity magnifying glass apparatus worn by the instructor with a floater-image-process ing apparatus on a three-dimensional image o- the visual field of the medical practitioner captured by 'he CCD camera of the three-dimensional d i g 11 a 1 •.•ntity magnifying glass apparatus worn by the medical practitioner shared in display by the three-dimensional digital entity magnifying glasses of both the medical practitioner .ind the instructor and, further, change a degree of transparency, between 0% and 100%, of three-dimensional visual instruction floater images due to an image chroma-key compos:te image-processing apparatus or convert: them so that thei:. tint may be judged and display a resultant composite irna- •:> on the LCDs of both the medical practitioner and the insti ictor. By this three-dimensional digital entity magnify inc. glass system, the medical practitioner or the instructing medical practitioner can obtain an ideal camera angle when imaging arid observing a s u r gic a 1 fie1d w i t ho 111 taking his eyes off the s u r gi c a 1 field and without the head of t.:e medical practitioner blocking
his view and, the re f o re, •; n concentrate on a surgical operation without being worried about a camera position.
Of course, this act .-jr. of instruction can he instructed by the in,-.; t r uc' :;r at a location away from an operating environment", and so is performed without being perceived by a patient so that he may not be mentally influenced unnecessarily; moreover, the medical practitioner not only can be trained by the instructor through a simulation havi::-; a reality by expressing, in a working space over images .->f an entity (patient) on the LCD of the medical practi t .tone , also a three-dimensional depth of a three-dimensional irru. ••• of the same instrument as that actually used by tho medi : 1 practitioner operated by the instructor in a rnedjcnl i .petitioner's visual field image virtual space of the .instrv 'tor's three-dimensional digital entity magnifying glass to. the entity before c. he eyes which is captured on the tree-e; mensional digital entity magnifying glass visual I old screen worn by the medical practitioner but also apparently can generate visually an operating situation in wh .. ch he operates himself by using the same instrument, sc th . t even poor verbal communication between the medical pract tioner and the instructor can be sufficiently compensated ror visually.
By conducting thus practical educational training services involving visual instructions, time-wise expenses involved in transportatio: of one a plurality of instructors
(groups of lecturers) can be eliminated, thereby accommodating an emergency operation even at a remote location.
Further, in a surgical operation, a student (customer) can be trained across the border without transporting a patient, from arbitrary operating facilities and without: medical licencing regulations even if an instructing medical practitioner (lecturer) of an educational program and th student (customer). By further improving t: h i s three -c irne i..--; ional digital entity ma gnifying glass apparatus to compos: tely display three-dimensional image information obtaino i through image p ro ce ^ s i ng . con ve r s :i c r: by 'hanging a three-dimensional anatomical image of the bo .e, the blood vessel, thy nervous tissue, etc. created beforehand from information of a t:omogram such as a CT pict'.re or an MR I picture of a patient on whom an operation is to be performed or a surgical stent image into which the ,:mai anical three-dimensional perspective image are three-dimensiona11y composed together with a dissection or bone cutting position similarly by a percentage of between 0% ..nd 100%, especially, changing a degree of transparency or , color tone of a site or a tissue indicative of an anatomical landmark or giving a visual difference to the medical ractitioner's three-dimensional digital entity magnifying glass CCD image information gh! wireframe disolav etc. so as to facilitate
d i s c r i m i n a t i o n , t h o n, o d i c a p r a c t i t i o n e r c a n v i s u a 11 y w a t c h shapes of the blood ve s s e ; .•; and the bones as if by seeing through the skin or the s c ••. t tissues and further watch how the nerves run through, th. bones as if seeing through the bones and, therefore, can safely perform a surgical operation accurately. How- of the surgical field captured by the medical practitioner's CCD camera, based on a change in position of the head o i' the medical practitioner arid a change in posture of the patient. To make such a configuration clinically applicable, still images are taken out from a medical practitioner's visual field image picked up by the two mutually synchronized right and left CCD cameras of the medical practitioner's three-dimensional digital entity magnifying ;lass and a reference point is set to three positions on an anatomically universal hard tissue such as a tooth or a bone ,-\ each of these images, position information of a total of -ix points, three points each, is recorded. By recording ;:. three-dimensional positional
relationship between a medical practitioner and a patient through three-point measurement by use of these two right and left CCD cameras and following positional movements of the three points that provide a reference in a medical practitioner's visual field image (moving image), it is possible to always grasp ,;: three-dimensional positional change in the three-dimensional digital entity magnifying glass and a posture 01 tlv. patient. At the same time, by accurately plotting the s line three points as those that provide a reference over -.. n anatomically universal hard tissue on a three-dimensional anatomical image or a surgical stent image on three point.-:-: of the hard tissue of s t i ] 1 images taken out from information from the two CCD cameras of the medical practitioner's th/ee-dimensional digital entity magnifying glass and accurately superimposing the three-dimensional anatomical image or the surgical stent image in an image of a sui-:ical field, a three-dimensional position, in a three---dinu: MS ional image of the surgical field, of the three-dimensional anatomical image or the surgical stent image is recorded in a space of a space of the surgical field captured by the medical practitioner's three-dimensional digital entity magnifying glass. It is thus possible to three-d .i mensiona lly follow the three-dimensional anatomical image or the surgical stent image as three-dimen.s iona.1 positions move of the patient and
the medical practitioner':- "hree-dimensional digital entity mag n i. f y i n q g 1 a s s .
These 1; hree -diinens . jnal anatomical image and • surgical stent image can ;,e managed and operated in a consolidated manner ar a romote location. For example, in an emergency operation re' uired in an area far away from a family doctor, a meci i ca 1 practitioner can transmit with various types of comrnunic .tion means information, of a patient, just about to be Further, in the present system, not only a
face-to-face combination of a medical practitioner and an instructor but also a plurality of instructors can
simultaneously display a three-dimensional anatomical image or a surgical stent image or participate in three-dimensional visual • Similarly, by transmitting three-dimensional image information captured by CCU cameras of the three-dimensional digital entity magnifying glass CCD mounted on a medical practitioner through communication means such as the internet or an intranet a.id accumulating it in a server, a plurality of person;- on v.'hich mutually synchronized three-dimensional dxgita magnifying glasses are mounted respectively can sh,i i e that image information by using communication means t,uch as the internet or the intranet. The relevant images, which have been accumulated moving images though, are of a visual field of a surgical field obtained when a medical r-ractitioner actually performed a
surgical operation and so r in be such as to have an extremely high degree of reality w.i : h a feeling of sensation1'. Moreover, by permitting e :oh of these multiple viewers to arbitrarily make swi'.. chov r between a display mode and a non-display mode of image; accumulated in the server together with surgical field image information shared by these viewers such as throe-dimensional anatomical images of the arbitrary bones, blood vessels, and nervous tissues produced on the basi-1 of typographic information such as CT o r M RI of a p a t i e n t, i" o b <. operated the a n t o m i c: three-dimensional perspective images and surgical stent obtained by accurately composing three-dimensionally position of dissection bone cutting using an image position-correction follow-up apparatus it is possible to demonstrate operation with more information. in addition removing f1oater-image-processing background iiruge from viewer visual field utilizing ccd cameras diciita entity magnifying glass extract only nree-dimensional pointing device various types instruments identical those used medical practitioner chc.nge degree transparency percentage between ai-d provide floating whose tint has been chan for each viewers chroma-key accumulated> due to the three-dimensional digital entity magnifying glass mounted as viewing means for each of the viewers in condition where each of the viewers can switch between the display mode
a n d t he n o n - d i s p 1 a y ru o d e arbitrarily, it is possible to carry out bidirectional discussion or operation simulation on a three-dimensional moving .image or still image by utilizing a communication line such as the internet.
Further, in a case where a surgical operation is di s 1.1: i b uted and publisher to third parties at a remote location by using vai Moreover, when an instructor or a assistant is selected by using extremal large amount of information and a three-dimensional digita. entityraagnifying glass capable of inspection, instruction, and recording and in virtual experiences of an operating situation of a medical practitioner, it is pus sib. When a monetary contract is signed with such a registered medical doctor or technician on the network and if breach of contract occurred on the side of a medical practitioner or an instructor in training of a surgical
operation or helping, such a system can be configured that resultant damages may be charged in accordance with contents of the contract already signed; accordingly, by this contract already established upon application, which has been, signed by the medical practitioner and the instructor not acquainted with each ^ther though, the well-qualified person (instructor) awaiting at a remote location or the medical practitioner can accept a temporary staff at ease and, m o r e o v e r, a manager • > f the network who promotes a procedure for this application can cause beforehand the medical practitioner co s.-lect second-choice and third-choice persons of tha instructor (assistant) so that essential damages on the ide of the medical practitioner
may be compensated for, t) oreby avoiding a trouble through
»
discounting in accor fiance with skills of the instructor in
some cases .
Further, if an unexpected situation such as an.error in medical treatment occurred in an action of operating involving an instructor (assistant) using a three-dimensional digital entity magnifying glass system,
*
it is of course possible to easily clarify an erroneous position based on a recoid of information by the three-dimensional digital entity magnifying glass and also to give variations in scope responsibility and guide fee to contents of the contract beforehand in accordance with a degree of difficulty of *-. a c h of operations or a degree of
involvement corresponding to each of technical fields of the instructor or the as.-;;:. stci.. t and clarify the responsibility range corresponding to a training fee presented by the medical practitioner •n a cordance with the contencs of the contract; it is thus possvole not only to disperse risks of compensation for an ecroi in medical treatment or a defect but also to add such factors as skills of the medical practitioner, which provide criteria for insurance fees of a typical term insurance . :nd also a degree of difficulty of a case as well as skills • - persons) for each operation so that more essentially streamlined setting of insurance fees may be performed, which basis for calculation of the insurance fees can be improved in terms of accuracy by accumulating experiences entity magnifying glasses of identical standards and, further, can record the situation in a recording medium so that the skills may be evaluated objectively. Therefore, in the conventional case of auditing an action of medical treatment in a medical treatment field, conventional contents of auditing on 1 •/ matching between diagnosed contents and the action of treatment can be expanded further to objectively evaluate ;-. surgical field (personal skills of medical doctor), theri !:>y improving not only the skills of each of the medical doctors but also technical levels as a whole.
In a surgical opei ;tion actually performed by a medical practitione.::, of course, a three-dimensional anatomical image eo.npose>-. and displayed in a medical practitioner's visual field image, a surgical stent image, or an instructor image o: a plurality of instructors interferes with a'visual ;ield image commonly owned by the medical practitionei and i he instructor, so that by enabling a display mode operation and a non-display mode operation with switching that will , i o t block operations of using the hand or foot involved in an action of operating by the medical practitioner or the instructor through wishes of the medical practitioner or the instructor, it is possible to carry out a safe operation by utilizing features of the three-dimensional digitaJ entity magnifying glass as a pure
digital magnifying 'past; without blocking a view of the me d i c a 1 p r a c t i t i o n e • .
Brief Description o : the Drawings
FIG, 1 is an i 1. 1. u s i : •,; t i o n showing a configuration of a three-dimensional iiigi: il entity magnifying glass;
FIG. 2 is a conceptual illustration of performing of three-dimensional iiu: tru-• cion by use of chroma-key composition of a flruitin- image by utilizing three-dimensional digital entity magnifying glasses of identical standards,
FIG. 3 is a concepr.ua! illustration of performing communication by utilizi:...; the internet with the digital entity magnifying g;..:.-.ss i: FIG. 1;
FIG. 4 is a concept .,11 illustration of a plurality of instructors carrying out d configuration of FIG. 3;
FIG. 5 is a conceptual illustration of
three-dimensional positi FIG. 6 shows how anatomical images can be composed and d i 5 p1ay e d; and
FIG. 7 is a diagram .lowing a flow of image information in the present system.
Best Mode for Carrying out the Invention
An embodiment of the present invention will be described below together with illustrated examples.
Throughout the drawings, the same symbols indicate the. same components. FIC. 1 is an illustration showing a basic configuration of a three-dimensional digital entity magnifying glass 1. As shown in FIG. I, a right-side LCD36R (monitor) and a left-sid. LCD36L installed in the three-dimensional digitaj entity magnifying glass 1 have a spread that contains the espective right and left eyes of a medical practitioner 3, which may be an instructor or a person to be ins true1 ted t:,ud mounted on the head (face) by using a headgear 37. A ;ight-side CCD camera Ic-R and a left-side CCD camera ic-L are provided integrally with the three-dimensional d;cjita.!. entity magnifying glass 1 to a position that nearly corresponds to a central section between the right and lef. pupils in condition where their three-dimensional diigita. entity magnifying glass 1 is mounted on the face. It should be noted that image information from each of r.hese two CCD cameras Ic is used to give synchronized individual images on right and left CDs, thereby providing a t hre<:- image with little uncomfortable feelin. in m the same manner as condition where an optical magnify: ug glass is used.> Preferably the-- CCD >.-amera Ic is an automatic focusing CCD camera equipped with ,-i zoom lens. This type of camera is automatically adjusted to give appropriate focusing for
an arbitrary magnif •. of a wor ker and a work site, so that a medical practitioner can minimize muscle fatigue of his neck owing to fixation of the headgear-mounted head.
In this configuration, as shown in FIG. 2, by utilizing a system or su :h a three-dimensional digital entity magnifying g.. ass to project images due to image information of each of the right and left two CCD .^cameras Ic on the medical practitioner 3 to the respective LCDs of the three-dimens ion.-;': dini.tal entity magnifying glasses having identical str;j..dar\'3 and mounted on the medical practitioner 3 and an instructor 12 so that they may share a three-dimensional visual field image (image of an inside of the mouth cavity c f a patient 32) . On this shared medical practitioner's visuc-.j. fie-Id image, a virtual three-dimensional i.:.;.tru: t, ion mark (virtual image! 30a is chroma-key composed vhicli is obtained by imaging a three-dimensional instruction mark 30 in front of a monochromatic background plate 31 by utilizing the right and left CCD cameras Ic of the three-dimensional digital entity magnifying glass 1 mcuntee on the instructor 12 and floating i I: . Thiy t; h rcie-i.i :i m. nsioi. 11. instruction in ark 30 can bo displayed as a line drawing in a visual field image of the medical practitioner 3 by showing a trajectory for a constant lapse of time or d.i.^;?lay> -J as a still image.
Accordingly, the present system, which is comparatively simple but uses the CCDLCD monitors of identical standards, permits the instructor 12 to accurately give a three-dimensional image instruction to the medical practitioner without performing complicated correction. That is, the medical t'lacti tioner 3 can provide training with an extremely natural posture almost identical to that in a case where he actually performs a surgical operation using his own hands, different from such movement.1:; as those for giving instructions en a computer monitor conventionally. Even, in a case where this three-dimensional in:-, t ruct ion mark 30 is replaced with an actual operating instrument, the instructor 12 can perform in a medical practitioner's visual field virtual space che same manipulations as those in the case of actually pei forming a surgical operation to a surgical field developed on an LCD screen of the three-dimensional digital entity magnifying glass 1 mounted on himself. Similarly, '• he medical practiti.oner 3 can experience manipula' ;ons of an instrument having a three-dimensional depth in such a situation a.s if he himself were performing a surgical operation. Furthermore, by changing t. degree of transparency between 0j and 100% of three-dimensional visual instruction f-loating images of the instructor 12 which are composed and displayed in a visual field image of the m-dicci 1 practitioner 3 or, in addition to it, using a chroma-key composite image-processing
apparatus capable of tim conversion, it is possible to easily distinguish bvtween a real image and a virtual image, thereby avoiding a confusion on the monitor screen even in a case where the med i. oal Similar to thi-; chr to accurately superimpose such three-dimensional anatomical images and surgical stent image layers into the visual field image of the medical practitioner 3 in such a manner as.to be equally scaled do-./ri on the LCDs of the-three-dimensional digital entity magnifying glasses 1 mounted on the medical practitioner 3 and 1 he instructor 12 shown in FIG. 1 as a transparent image such as shown in FIG. 6 and, at the same time, perform three- dimensional position correction in them. In such sca 1i :i• \ correction and position correction, in a case where it :; : difficult to directly obtain a hard tissue reference poi!:C depending on a site, a point marker such as lead is stud- co three different fixed points on the skin which are visible from the CCD cameras of the three-dimensional d^itaJ entity magnifying glass 1 at the time of performing n /.urg^cal operation during taking a CT or MR I picture and di :, play .'U on a three-dimensional surgical stent so that the corresponding on-the-skin different three-point markers on the respective still images from the right-side CCD camera Ic-R and the left-side CCD camera Ic-L for displaying a surgical field image of the three-dimensional digital entity magnifying glass 1 may agree with each oth>-i:, thereby accurately performing position correction. In this case, although it is difficult to perform position <. orrection and equal-scaling correction simultaneously on t im ri left lcds in the three-dimensional digital entity magnifying glass of> present invention, .:. d t \. •> n conducting position correction (super-imposition) or r.he • rie-side LCD monitor in different two directions, it :•. p o.'.;.; i b 1 e to easily record position information. Further, if i site to be operated is the head, especially, covers t).•>.. upp • r and lower jaws, three different reference points are. ->bta ned for each of the si ten and two different surgical , i ent;-; are prepared, to automatically follow the respective floating surgical stent images. A three-dimen.s iona i. anatomical image layer or a three-dimensional si.;gicai stent layer image, if it has undergone scaling correction and position correction once, is composed into a mescal jractitioner's visual field image and displayed by caj.urinq a displacement of a three-dimensional positional measurement reference plane 35 made up of three hard tissue reference points 34a-34c on the tooth in the tooth rev 33 a s image information for the right and left two CCDs Ic chariq- to automatlca -y perform three-dimensional position correction thereon ia real time. In this case, a cnange in posture of the patiei.t may be captured by utilizing an image as described above c> mounting a light emitter such as an LED to the three different points attached on the universal tissue of the head or the patient in order to more clearly display the reference points on the image. Alternatively, a gyro mechanism may :>a mounted to each of a fixed point such as the patient's ha:ii tissue or the three~dirnen.°, j. ona 1 digital entity magnriiying glass mounted on the medical practitioner to reo.-vd a change in three-dimensional position of the pat:it-nt and the medical practitioner's three-dimensional, djaita! entity magnifying glass with respect to the fixed point (reference point) and accurately align it with the reference point on a still image due to information from the t ight and left CCD cameras obtained from the medical practitioner' .ci three-dimensional digital entity magnifying glass so r hat. an initially-corrected three-dimensional anatomical image or three-dimensional surgical stent image inay be followed three-dimensionally as the patient's postmi- changes or the head of the medical p r a c t11 i o n e r move s.
By improving :, his system, it is possible to share medical practitione;'s images among a plurality of instructors as shown in FIG. 4 so that medical specialists may give an appropriate advice. This advice is shared by
the medical practitioner ; and all of the instructors 12 by caption ing a signal rrem vital sign measuring instrument 5 at a peripheral of V n ••; LC. > monitor of the three-dimensional digital entity magnifying glass I in such a manner as not to damage an image of a surgical field, so that it is possible or the instructors not only to provide an instruction as grasping a field situatior, more accurately but also for the medical practitioner 3 to concentrate on a surgical operation without taking his eyes off the surgical field to the monitor etc.
Further, in t no ca-.e of transmitting through a communication network sucr; as the internet image information obtained by providing identical image as that on the three-dimensional digital entity magnifying glass with advices and comments due to writing of a line drawings or a three-dimensional pointing instruction, to give an advice and a comment by ust of pointing of the three-dimensional pointing mark 30, a me1- ing image can be converted into a still image once and provided with the writing or pointing instruction as it is so that a resultant composite image of this still image may be distributed to terminals of a plurality of studen;;1 to be instructed. Alternatively, conversely, in a case- where a medical practitioner is helped with a surgical ope.ro t. ion by an instructor at a remote location as receiving a w rit ing or a pointing instruction on his monitor, a moving image distributed to the side of
the instructor is converted into a still image once and provided with a writing or a pointing instruction so that a resultant composite image thereof may be distributed to the monitor of that, medical practitioner.
As permitted, such c. wiring or a pointing instruction can be given also to i-i moving image, in which case a moving image is once converted into a still image and then effectively returnee to t':>e moving image after dividing an LCD screen of the three-dimensional digital entity magnifying glass 1 sc chat the still image may be separately displayed into a visi '-.I £:i'-Ld image. It should be noted that if a medical practitioner o. tc. serves as an instructor, such a pointing instruction or writing is typically given by a third party 1 i v e - b r c :-:i dcasting a surgical operation by the medical practitioner or an instructor in charge in a service company of an educational ,;- ystem having the server 8 therein. Although a method for converting a moving image once into a still image and then returning it back to the moving image after carrying out a pointing instruction etc. as described above is known and so not detailed; in short, according to this method, an instantaneous data image from a CCD camera etc. is once accumulated in an image memory and then read and, simultaneously, an image of a pointing instruction etc. input from an input unit (which may be a moving one .as well as a still one, but in .:ome cases where a pointing instruction is reeved during exp i..'. nat.'. on and training,, preferably the
point image etc. is a mov ng one) is fetched so that these images may be 'composed u;::i.ng a known method such as layer processing or composition in a memory. It is thus possible to share information of , medical practitioner's three-dimensional image jor a three-dimensional digital entity magnifying glass accumulated in the server 8 and information of a surgical stent image for composing and displaying a three-dimenilonally accurate equal-scaled three-dimensional transparent image into a surgical field image among viewers 7a-l through 7a-n respectively equipped with a plurality of synchronized three-dimensional digital entity magnifying glasses by utilizing a communication line such as the internev, thereby performing discussion on a three-dimensional moving image or a still image through bidirectional communication. In such communication by use of all of these images, from a viewpoint of patient's privacy protection, all of communication digital signal data is indicated on the three-d.imensional digital entity magnifying glass 1 through an audio/visual decoder 4; therefore, in the case o: hacking by use of an ordinary personal computer terminal, having a hardware decoding unit, a risk of information leakage is low.
With this, it is possible to transmit to the educational program distribution company server 8 from the terminal personal compute,: 6 connected to the internet image information obtained from this digital entity magnifying
glass 1, image information such as values and graphs of an illuminance, a temperature, a humidity, a pressure, and a speed that represent an environment of a situation of a vital sign of a patient in the case of a surgical operation and a situation of each technical operation in the case of the other precision techniques, voice information obtained from an audio microphone mounted on a medical practitioner, and visual instruction information such as the three-dimensional ii. s t rut t: i on mark 30.
The information received by the educational program distribution company server 8 is either directly distributed to the terminal personal computer 6 via the internet 9 or distributed in condition where image processing or interpretation in accordance with each language is performed thereon by the educational program distribution company server 8. Similarly, it is possible to transmit to the educational program distribution company server 8 from the terminal personal computer 6 connected to the internet voice information obtained from an audio microphone Ib of the terminal personal computer 6 and visual instruction information such as r.he three-dimensional instruction mark 30 .
The information received by the educational program distribution company server 8.is either directly distributed to the terminal personal computers 6a-l through 6a-n via the internet 9 or distributed in condition where image
processing or inte r; » e ta ' Ion in accordance with each language is performed thereon by the educational program distribution company server 8.
Next, a method for distributing information in the above-described distribution system exemplified in FIG. 1 is described along the figures in divided cases of the medical practitioner 3 (instructor) training the auditing student 12 (customer) as shown in FIG. 3 and specialized instructors (groups of lecturers) training the medical practitioner (customer) as shown in FIG. 4.
In this case where a medical practitioner
(instructor) 3 performs a helping and training/educational program for a surgi• • o1 operation and other precision techniques performed under direct observation on a student (customer) as shown in FI'">. 3, data of the vital monitor 5 or the precision teehniqee performing environment measurement instrument 5 and information from the audio/visual decoder 4 are composed with each other by the terminal personal computer 6 and superimposed through the audio/visual decoder 4 on a magnified image of a surgical field which is obtained from the three-dimensional CCD camera Ic of the three-dimensional digital entity magnifying glass 1 of FIG. 1 mounted on the medical practitioner 3 and which is projected on thi- eyeglass type LCD monitor Id mounted on the head of the instructing medical practitioner (leetu re r) .
Image information vv'hich is projected to the LCD monitor Id and oral instruction information for the microphone Ib are distributed by a specific contractant student (customer) to the terminal personal computers 7a-l through 7a-N direct:y or to the technique-assisting and as s i s t ing-and training/educational distribution system distribution company server 8 from the terminal personal computer 6 by utilising the internet 9. If the medical practitioner (instructor) and the student (customer) speak different languages, in distribution of an educational program, this information is consecutively or simultaneously interpreted into the language spoken by the student (customer) 3 and transmitted to the student (customer) by the technique-assisting and assisting-and training/educational dis -.: ribution system distribution company server 8. Accordingly, a lot of specific contractant students (customers) can yet a virtual experience with such realistic sensations that, he feels as if he were in a field of an opeiating situation about a surgical operation and other highly precisiu-n oriented techniques distributed to the terminal persona] computers 7a-l through 7a-N.
Since the communication configuration can be utilized
in a bidirectional manner, questions etc. of a student
(customer) are verbally input with the microphone Ib of the
terminal personal computers 7a-l through 7a-N so that they
may be transmitted in a reverse direction to the server 8
via the internet 9 and organized to eliminate duplications and then similarly ;:. iansir.itted to the earphone la of the instructing medical practitioner (lecturer) from the technique-assisting and a s s i s t i n g - a n d
train ing/education a.", distribution system distribution company server 8 by utilizing the internet 9.
An instructing medical practitioner (lecturer) 3a inputs his answer to a question of a specific student (customer) into the t e rrrii :: a 1 personal computer 6 a by using the microphone Ib, to transmit it via the internet. 9 to the technique-assisting c-rnd , .; sis ting-and
t raining/educationaJ d i s\ : i b u t i o n system distribution company server 8. Eurther, if the student (customer) and the medical practitioner (instructor) speak different languages, the educational program distribution company server 8 consecutively or simultaneously interprets the language spoken by the in : tructor into the language -spoken by the student, (customer) .:.nd distributes it via the internet 9 to the terminal personal computers7a-l through 7a-N of the students (customers).
Conversely, if specialized instructors (groups of lecturers) at a remote location guide the medical practitioner (customer) .'. b as shown in FIG. 4, magnified surgical field image information A obtained from a two-dimensional or three-dimensional CCD camera Ic of the digital magnifying glass of FIG. I mounted on the medical
practitioner (customer) 3, vital sign data .B superimposed on a magnified surgical field image or image information B from a highly precision oriented technical operating environment measurement instrument, and a question C from the customer medical practitioner input through the microphone Ib are input through the audio/visual decoder 4 into the terminal personal computer 6 and then transmitted therefrom to the technique-assisting and ass isting-and training/educational distribution system distribution company server 8 via the internet 9.
If a medical practitioner (instructor) 3b and the specialized instructors 'groups of lecturers) 1 ^speak different languages, the educational program distribution company server 8 consecutively or simultaneously interprets the ].anguage spoken by tho medical practitioner into that spoke 11 by the specialized Instructors (groups of lecturers) and transmit it to the specialized instructors (groups of lecturers) 7 . In t:;is case, it is transmitted via the internet 9 to terminal personal computers 7b-l through 7b-N of one or, if technical contents cover multiple fields, a plurality of specialized instructors (groups of lecturers).
As observing magnified surgical field image information A of the ir.edj.cal practitioner (customer) 3 displayed on the terminaJ personal computers 7b-l through 7b-N and vital sign data B superimposed on a magnified surgical field image or image information B from the highly
precision oriented technical operating environment measurement instrument, t ::,; specialized instructors (groups of lecturers) 1 perform technical training and a q u estion-and-answer session by inputting visual instructions by using th<: three-dimensional instruction mark and oral instructions by using the microphone ib utilizing terminal personal computers through so that this input information may be transmitted in a reverse direction to tho technique-assisting training distribution system company server via he internet further medical practitioner internet.> If the specialized instructors (groups of lecturers) 12 and the medical practitioner (customer) 3 speak different languages, the educational program distribution company server 8 consecutively or simultaneously interprets the language spoken by the instructors into that spoken by the medical practitioner (customer) 3 and directly sends information of the three-dimensional instruction mark 30a onto the LCD monitor 36 of the three-dimensional digital entity magnifying glass of FIG. 1 mounted on the medical practitioner (customer) and information of the oral instruction to the earphone la. It is thus possible to perform bidirectional communication on a specific contractant medical pracv. itioner (customer) 3, thereby providing skill educational training services.
Further, in FIG. I, the terminal personal computers 6 and 1 are connect€jd to t.he internet 9 and, therefore, of course are equipped with a function to access a homepage of the educational program distribution company server 8.
In such a mann••?:., wl en an auditing student (customer) buys technique-assisting >md train ing/educational programs by utilizing the homepage of the educational program distribution company server 8, he can search this homepage of the educational program distribution company server 8 for general advertisings such as histories of instructing medical practitioners (lecturers) of each of the programs, technical contents and technical fields, languages spoken by them, instruction time, and a tuition and also popularities and evaluations of students (customers) who have attended lectures already in real time, thereby obtaining unilateral promos of the instructing medical practitioners (lecturers) or objective criteria other than subjective evaluations or acquaintances who have participated in the same educational program already. Similarly, the students (customers) can apply for receiving lectures of the educational programs and pay tuition on the homepage of the educational program distribution company se rver 8 .Further, an instructing medical practitioner (lecturer) who teaches ai educational program can utilize the homepage of the educational program distribution company
server 8 to thereby give promos, accept an application for lecture reception, a'id confirm collection of the tuition for formal acceptance.
Conversely, in a case where a medical practitioner (customer) who wishes for receiving a lecture buys an individual technical/educational assistance and training/educational program by a plurality of field-specific expert instructors (groups of lecturers), he can search a web site of the internet homepage of the educational program distribution company for publicized technical contents and technical fields of the instructors (groups of lecturers), languages spoken by them and instruction time, arid vailous operation-unit costs for individual technical education/training, thereby reversely nominate any one of the instructors in accordance not only with unilateral promos ot the instructors (groups of lecturers) or evaluation criteria due to subjective evaluations of acquaintances who have already received a lecture of the educational program but also with objective criteria .
Moreover, tuitions are clearly published in the internet homepage web site of this educational program distribution company so !;hat no one needs to directly negotiate about tuitions .md, further, instructors (groups of lecturers) for various technical fields are organized by the educational program distribution company into an
appropriate team so that the customer can get a high degree of training. Also, a medical practitioner (customer) can pay for educational training services at the internet homej)age web site of this educational program distribution company.
Moreover, the instructors (groups of lecturers) can automatically receive lecture fees determined by the education program distribution company in accordance with the number of the participant instructors (groups of lecturers! for the skill educational training service lectures charged in unit:; of operating contents or time relating to a surgical operation or any other highly precision oriented Techniques, thereby keeping themselves free from troublesome ofrice procedures.
Further, although the above description has been made on the assumption that th^ internet would be used, besides the internet the present invention can be applied also to communication means that connects a medical practitioner (or operator of precision woik) and a party who receives information therefrom or receives it and transmits it bilaterally to each other by utilizing a closed network such as a intranet or any oth^r dedicated line. Further, applicable communication method may use a communication satellite and any other means appropriately selected.
The present three-dimensional digital entity magnifying glass system which can enable the third party to
experience virtual cp^rat ::>ns has extremely large amount of information and saff- and Moreover, the present system is of a real time configuration and, at the same time, enables communication only by means of three-dimensional digital entity magnifying
glasses having identical standards and, therefore, makes it very difficult to tamper data, has an extremely high degree of authenticity, and enables observing a medical practitioner's operating situation in virtual experience so that skills of the inedic.il practitioner and existence of defects can be assessed oojectively at a remote location.
An instruction image to be composed in a medical practitioner's visual fi-.rld image and displayed or an * anatomical three-dimensional image or surgical stent image to be composed in a medical practitioner's visual field image will not block manipulations by use of the hand or foot involved in an action of a operation by a medical practitioner or an instructor if the medical practitioner in the case of his leadership and the instructor in the case of his leadership will use an eye-contact switch or an audio command switch.
FIG. 7 is one example of a schematic diagram showing a situation of communication through the internet and by use of the above-described three-dimensional entity digital magnifying glass 1. As snown in this figure, visual field image information 40 from a medical practitioner's CCD camera input to a medical practitioner's personal computer is transmitted using the internet and chroma-key image-composed (42) with an instructor's CCD camera's floating image 43 transmitted from an instructor's personal
computer via the incerne', to display chroma-key composite image information 42 on •, medical practitioner's LCD.
Similarly, the ins i: ructor can provide, in the instructor's personal computer, chroma-key composite display obtained by composing the visual field image information 40 of the meuical practitioner's CCD camera transmitted through the Internet displayed on the instructor's LCD and a fJoater-image processed image
t
obtained from instructor's CCD camera information 41, to thereby perform training and instruction on the instructor's LCD with such a reality that as if one were performing a surgical operation by using medical apparatuses actually used. In the present syste.M utilizing the three-dimensional digital entity magnifying glasses 1 of identical standards, by performing chroma-key composition processing on this instructor's CCD camera floating image 43 in the medical practitioner's personal computer via the internet communication line, it is possible to reflect the sarrre image as the instructor's LCD on the medical practitioner's LCD without correcting a posit: ion of the instructor's CCD camera's floating image 43 on the medical practitioner's CCD camera's visual field image information 40.
In addition, by taking information from the medical practitioner's CCD camera into a server,computer, transmitting a surgical stent floating image 44 which has undergone position correc ion and automatic follow-up
processing to the medical practitioner's personal computer and the instructor's personal computer, and performing chroma-key composite image processing on rhe visual- field image information 40 of the medical practitioner's CCD camera, it is possible to reflect the surgical stent image on the LCDs of the medical practitioner and the instructor. Since those manipulations are all performed on the basis of the visual field image information 40 of the medical practitioner's CCD camera, the three parties can share the same images without performing position correction. As can be seen from this schematac diagram, the medical practitioner' s three-dimensional digital entity magnifying glass system and the instructor's three-dimensional digital entity magnifying glass system have totally the same functions, so that the instructor and the medical practitioner can be switched with each other easily.
It is to be noted that a configuration and a communication subject of a three-dimensional digital entity magnifying glass system of the present invention are not limited to the above-described embodiment and of course can be changed variously wichout deviating from the scope of the present invention.
Industrial Applicability
A three-dimensional digital entity magnifying glass equipped with an image composition three-dimensional visual
training function and a sechnical-assistance and training/educational distribution system of the present invention relate to a thiee-dimensional digital entity magnifying glass apparatus for performing technique training by utilizing a communication network such as the internet. Mora s p e c L f i c a :. .1 y , the present invention relates to distribution of a highly precision oriented specialized technique educational program for a surgical operation performed under diroct observation of a human being or any other manual operation and a three-dimensional digital ervciiy magnifying glass icr education/training for a surgical operation perfoii.ied by a specific customer or any other manually-operated highly precision oriented specialized techniques (unat is, such a surgical operation or other operations is controlled through visual instructions or oral instructions by use of three-dimensional image composition based on information obtained while a visual field of a person with no technical knowledge assumed to be a robot from a remote location) and, further, enables building up a stadium system that enables chroma-key composition wii.hout using large-scale facilities even at a remote location by a medical practitioner and an instructor using three-dimensional digital entity magnifying glasses of identical standards and, at the same time, provides means for recruiting students and instructors for various kinds of educational programs and
on-the-internet settlement of instruction fees for more inexpensive technique instruction and tuitions on the students.
Generally, in an educational program for live-demonstrating a surgical operation or any other manually operated specialized technique, in the case of instructing a student to perform a highly precision oriented technique such as a surgical operation by an instructor, if the technique and the surgical operation on a closed site are to be observed from identical ideal visual field as the instructing medical practitioner (lecturer), to permit the plurality of students (customers) to sequentially observe an operating situation a'- each step of the surgical operation, not only extremely much of time is required but also the number of persons who can observe the operation simultaneously is limited by the physical constraints of the facilities used for the operation, and such lectures have consequently been observed by very few people. In addition, if the instructing medical practitioner (lecturer) and the student (customer) speak different languages, an interpreter needs to be present during the surgical procedure, so that a fee per student (customer) for participating in such an educational program has been expensive.
Further, in the case of a surgical operation, it is difficult to transport a patient himself to a remote location
for an educational program; also, in the case of other special techniques performed by hand which require large facilities or unmovable instruments, a visiting student (customer) used to arrange his schedule so as to fit in with the operating date and spend much transportation costs and time required by transportation. Moreover, when a person who wishes to receive a leccure buys an educational program, he cannot but discuss whether to buy it based only on unilateral promos of an instructing medical practitioner (lecturer) or evaluation criteria due to subJ2Ctive evaluations of his acquai r tances who have already received the lecture of the same educational program and, further, must separately pay a tuition through payment medium specified by each of the lecturers. Conversely, the instructor (lecturer) who opens a course of the educational program used to provide promos by utilizing various kinds of media, accept application for reception of the lecture irrespective of whether the number of students (customers) is large or small, and confirm collection of the tuitions for formal acceptance again.
Conversely, in the case of training on a surgical operation or a practical training of highly precision oriented operation which is performed at a remote location, instructors (groups of lecturers) must go to a spot to require a lot of costs, so that various costs (transportation costs and time required by transportation} related to the
training and, moreover, the remote area requiring much time to go there could not easily accommodate performing of an emergency surgical operation; especially in the case of training by a plurality ot instructors (groups of Lecturers), the costs used to become even higher and an emergency surgical operation used to be even more difficult to conduct. Further, if the medical practitioner (customer) and the instructors (groups of lecturers} speak different languages, it used to be necessary to take an interpreter to a field of a surgical operation.
Besides, since conducting medical treatment overseas is prohibited by legal regulations on the medical licensing, so that it is basically impossible for the instructors (groups of lecturers) to open a course of a livu-operation visiting educational program in which the instructors (groups of lecturers) themselves perform a surgical operation and conduct skill training in the foreign country.
Furthermore, if a medical practitioner (customer; actually wishes to receive training of a surgical operation or any other highly precision oriented specialized techniques which are performed by hand, he cannot but select his instructors (groups of lecturers) within a limited human network and without clear criteria for calculating a tuition for each of the instructors (groups of lecturers), and he has to directly negotiate with them; therefore, he used to not only worry himself in selection of the instructors
(groups of lecturers) buT also have to pay the' tuition through the respective payment means separately in accordance with training ; ime and contents, thus resulting in troublesome office procedures.
Besides, when medical treatment is audited in the conventional medical fie.l'.ls, the present invention helps auditing consistencies between what is diagnosed and what is treated medically and i:; useful not only in decision of an error in medical treatment but also as criteria for calculating costs for medic.--! treatment in a field of medical care insurances. Similarly/ the present invention enables to objectively evaluate an academically surgical field (medical doctor's personal skills), so that a medical convention etc. can be held through real-time presentation on the internet, which is extremely useful in direct evaluation of competence of medical doctors, that is, establishment of criteria i'or selecting medical doctors accredited by the convention.



I claim:
1. A three-dimensional digital entity magnifying glass system incorporating three-dimensional visual training functions, the system used for performing assistance and training and education under direct observation, by a medical practitioner while an instructor is watching a medical practitioner's surgical field image shared by the medical practitioner and the instructor through communication secured by use of communication means by utilizing three-dimensional digital entity magnifying glasses, which are worn by both the medical practitioner and the instructor and constituted of a pair of LCDs positioned directly in front of the two eyes and by three-dimensional digital entity magnifying glasses of identical standards for indicating image information from a right-side CCD camera on the right-side LCD and image information from a left-side CCD camera on the left-side LCD of automatic focusing CCD cameras equipped with a pair of synchronized zoom lenses attached toward a visual field of the naked eye from a position that correspond to a central section between the right and left pupils of the worker wearing the magnifying glass in front of the three-dimensional digital entity magnifying glass, the system comprising:
a floater image-processing apparatus [[for]] removing a background image by extracting only images of a three-dimensional pointing device which is operated by the instructor to provide visual training from a visual field image of the instructor captured by the instructor-side CCD camera on a front face of the visual field in front of the instructor-side three-dimensional digital entity magnifying glass; and
an image chroma-key composite image-processing apparatus for composing and displaying a three-dimensional visual training float image, on which no background output is performed by the image-processing apparatus on the medical practitioner's visual field image captured by the CCD camera of the medical practitioner's three-dimensional digital entity magnifying glass shared by the medical

practitioner's visual field image captured by the CCD camera of the medical practitioner's three-dimensional digital entity magnifying glass shared by the medical practitioner and the instructor,
wherein by utilizing the pair of synchronized three-dimensional digital entity magnifying glasses of identical standards worn by the medical practitioner and the instructor, the instructor provides visual training which is three-dimensional in the medical practitioner's three-dimensional visual field by utilizing images of the three-dimensional pointing device operated by the instructor which images are extracted by floating image processing from visual field images of the instructor captured by the instructor's CCD camera and chroma-key composed and displayed in the medical practitioner's visual field image captured by the medical practitioner's CCD camera projected onto an LCD monitor of the three-dimensional digital entity magnifying glass worn by the instructor.
2. The three-dimensional digital entity magnifying glass system incorporating three-dimensional visual training functions as claimed in claim 1, wherein the image chroma-key composite image-processing apparatus varies a degree of transparency of a three-dimensional visual training image of the instructor by a percentage of between 0% and 100% or changes a color tone thereof and displaying it compositely, so that the medical practitioner can easily judge training image and in order to prevent the medical practitioner's visual field and an action of an surgical operation from being blocked visually as a result of overlapping of the medical practitioner's work situation visual field image and a three-dimensional visual training image of the instructor when the floater three-dimensional visual training image with no background, output from the image-processing apparatus, of the three-dimensional pointing device operated by the instructor is chroma-key composed on medical practitioner's visual field image information in a CCD camera of the digital entity magnifying glass worn by the medical practitioner and displayed.
3. [[A]] The three-dimensional digital entity magnifying glass system incorporating three-dimensional visual training functions as claimed in claim 2, further comprising

an image position correction and follow-up apparatus [[for]] compositely displaying three-dimensionally accurate equal-scaling three-dimensional perspective images into the medical practitioner's visual field image displayed on an LCD of a three-dimensional digital entity magnifying glass by automatically performing position correction always in respect of variations in a visual field angle of the medical practitioner with respect to a patient or to variations in postures of patients, by means of:
in an image obtained by varying a degree of transparency, by a percentage of between 0% and 100%, of image information from the medical practitioner's three-dimensional digital entity magnifying glass CCD displayed on LCDs of both the medical practitioner and the instructor;
three-dimensionally superimposing an item of three-dimensional image information obtained through image processing and conversion, by changing a three-dimensional anatomical image of an item created beforehand from information in a tomogram of a patient on whom an operation is to be performed or a surgical stent image into which the anatomical three-dimensional perspective images are three-dimensionally composed together with a dissection or bone cutting position similarly by a percentage of between 0% and 100% by varying a degree of transparency or a color tone of a site or a tissue indicative of an anatomical landmark, or by applying a visual difference to the medical practitioner's three-dimensional digital entity magnifying glass CCD image information through wireframe display [[etc.]] so as to facilitate discrimination, and onto the medical practitioner's three-dimensional digital entity magnifying glass CCD image at an equal reduction rate by utilizing a reference triangular plane obtained through three-dimensional positional measurement by use of the right and left CCDs of the three-dimensional digital entity magnifying glass and by using, as reference points, three points on non-deformable hard tissue arbitrarily determined from among the medical practitioner's three-dimensional digital entity magnifying glass CCD image information; and by
enabling the anatomical three-dimensional image to automatically correct a change in position of, and follow, the reference point of the medical practitioner's

three-dimensional digital entity magnifying glass CCD image information as the medical practitioner's CCD camera position changes or the patient's posture changes.
4. [[A]] The three-dimensional digital entity magnifying glass system incorporating three-dimensional visual training functions as claimed in claim 3, wherein a three-dimensional anatomical image of an arbitrary item created beforehand from information in a tomogram of a patient on whom an operation is to be performed, owned at a remote location or a three-dimensional surgical stent image in which the medical practitioner or a third party designed beforehand a position of a dissection or a bone cutting based on the anatomical three-dimensional perspective image is composed into the medical practitioner's three-dimensional digital entity magnifying glass CCD's image information at the remote location and accurately superimposed one on the other to provide the respective composite display images showing an apparently entity-see-through situation, positions of which images are automatically corrected by the image position correction and follow-up apparatus in real time as the medical practitioner's CCD camera position changes and the patient's posture changes, to distribute the images compositely displayed to a three-dimensionally correct position always to the medical practitioner and the instructor by utilizing communication means.
5. [[A]] The three-dimensional digital entity magnifying glass system incorporating three-dimensional visual training functions as claimed in claim 3, wherein by accumulating the medical practitioner's three-dimensional digital entity magnifying glass's medical practitioner three-dimensional image information and surgical stent image information for compositely displaying three-dimensionally accurate equal-scaling three-dimensional perspective images into the medical practitioner's visual field image in a server so that each of persons wearing a plurality of synchronized three-dimensional digital magnifying glasses and sharing the image information accumulated in the server can arbitrarily switch the surgical stent image information on the medical practitioner's three-dimensional digital entity magnifying

glass's three-dimensional image information between a display mode and a non-display mode and, further, can transmit to the server a three-dimensional visual training image obtained by utilizing the three-dimensional digital entity magnifying glass's CCD cameras worn by them, and compositely display it to the medical practitioner's three-dimensional image accumulated in the server, thereby making possible discussions through bidirectional communication on a three-dimensional moving image or a still image, by utilizing a communication line.
6. [[A]] The three-dimensional digital entity magnifying glass system incorporating three-dimensional visual training functions as claimed in claim 1, wherein by providing pluralities of synchronized three-dimensional digital entity magnifying glasses worn by the medical practitioner and the instructor and the floater image-processing apparatuses so that the medical practitioner and the instructor may share the images, the plurality of instructors simultaneously guides one medical practitioner three-dimensionally by using the visual training image or switches and displays visual field images of the plurality of medical practitioners as occasion demands so that one instructor provides visual instruction and guidance to the plurality of medical practitioners three-dimensionally through communication means.
7. [[A]] The three-dimensional digital entity magnifying glass system incorporating three-dimensional visual training functions as claimed in claim 1, wherein when distributing and publishing a training situation by use of the three-dimensional visual training image to a lot of students other than the medical practitioner and the instructor or to a third party institution by using live-broadcasting or video-recorded broadcasting via communication means, by encrypting output signals of two right and left channels output from one three-dimensional digital entity magnifying glass's CCD camera in hardware of the three-dimensional digital entity magnifying glass and by encoding it in hardware of the other three-dimensional digital entity magnifying glass, normal image display is disabled unless a terminal having a prescribed three-dimensional digital entity magnifying glass hardware configuration is used, to provide security, thereby making possible protection of the patient's privacy.

8. [[A]] The three-dimensional digital entity magnifying glass system incorporating three-dimensional visual training functions as claimed in claim 1, wherein by presenting conditions during the course of training as well as offering monetary rewards to an operation assistant on the network by using the three-dimensional digital entity magnifying glass system capable of virtually experiencing of the medical practitioner's operating situation, to sign a contract with a registered or technician to get the instructor dispatched thereto, so that if an unexpected situation during a surgical operation occurs, the unexpected situation can be clarified based on a record of information of the three-dimensional digital entity magnifying glass system and a responsible range is also clearly defined in accordance with contents of a contract determined beforehand based on a degree of involvement between contents of each operation and each technical field of the instructor staff and an instruction fee presented by the medical practitioner based on the contents of a contract, thereby spreading a burden of risks for compensations for the unexpected situation and defects.
9. [[A]] The three-dimensional digital entity magnifying glass system incorporating three-dimensional visual training functions as claimed in claim 1, wherein it is made possible to objectively asses skills of the medical practitioner at a remote location by using the three-dimensional digital entity magnifying glass system that enables virtually experiencing an operating situation of a medical practitioner and has authenticity because this system is of a real time type and also capable of communication only by simultaneously using three-dimensional digital entity magnifying glasses of identical standards.
10. [[A]] The three-dimensional digital entity magnifying glass system incorporating
three-dimensional visual training functions as claimed in claim 5, wherein it is made
possible to switch an instructing image compositely displayed in the medical
practitioner's visual field, an anatomical three-dimensional image compositely
displayed in the medical practitioner's visual field image, or a surgical stent image

between the display mode and the non-display mode through switching that does not block operations of using the hand or foot involved in an action of operating by the medical practitioner in the case of his leadership or through wills of the instructor in the case of his leadership.
11. A method performed in a system incorporating three-dimensional digital entity magnifying glasses which are worn by both a medical practitioner and an instructor and constituted of a pair of LCDs installed directly in front of the two eyes and used for performing assistance, and training and education under direct observation, by a medical practitioner while an instructor is watching a medical practitioner's surgical field image shared by the medical practitioner and the instructor through communication secured by use of communication means by utilizing three-dimensional digital entity magnifying glasses of identical standards for indicating image information from a right-side CCD camera on the right-side LCD and image information from a left-side CCD camera on the left-side LCD of an automatic focusing CCD cameras equipped with a pair of synchronized zoom lenses which are attached toward a visual field of the naked eye from a position that nearly corresponds to a central section between the right and left pupils of the worker wearing the magnifying glass in front of the three-dimensional digital entity magnifying glass, the method comprising:
a step of floater image processing, by removing a background image by extracting only images of a three-dimensional pointing device which is operated by the instructor to provide visual instructions from a visual field image of the instructor captured by the instructor-side CCD camera on a front face of the visual field in front of the instructor-side three-dimensional digital entity magnifying glass; and
a step of image chroma-key composite image processing, by composing a three-dimensional visual training float image with background output by the image-processing apparatus on the medical practitioner's visual field image captured by the CCD camera of the medical practitioner's three-dimensional digital entity magnifying glass shared by the medical practitioner and the instructor and displaying it,

wherein by utilizing the pair of synchronized three-dimensional digital entity magnifying glasses of identical standards worn by the medical practitioner and the instructor, the instructor provides visual instructions which is three-dimensional in the medical practitioner's three-dimensional visual field by utilizing images of the three-dimensional pointing device operated by the instructor which images are extracted by floating image processing from visual field images of the instructor captured by the instructor's CCD camera and chroma-key composed and displayed in the medical practitioner's visual field image captured by the medical practitioner's CCD camera projected onto an LCD monitor of the three-dimensional digital entity magnifying glass worn by the instructor.
Following new claims 12-16 are inserted:
12. A three-dimensional digital entity magnifying glass system, the system performing
assistance and training and education under direct observation by a medical
practitioner while an instructor is watching the medical practitioner's surgical field
image shared by the medical practitioner, the system comprising:
a plurality of three-dimensional digital entity magnifying glasses worn by both the medical practitioner and the instructor, wherein each three-dimensional digital entity magnifying glass comprises a right-side LCD, a left-side LCD, a right-side CCD camera on the right-side LCD, and a left-side CCD camera on the left-side LCD;
a three-dimensional pointing device, operated by the instructor, pointing images;
a floater image-processing apparatus removing a background image by extracting only images of the three-dimensional pointing device; and
an image chroma-key composite image-processing apparatus composing and displaying a three-dimensional visual training float image on the medical practitioner's visual field image captured by the right-side CCD camera and the left-side CCD camera of three-dimensional digital entity magnifying glass.
13. The three-dimensional digital entity magnifying glass system as claimed in claim
12, wherein each of the right-side CCD camera and the left-side CCD camera is an

automatic focusing CCD camera equipped with a pair of synchronized zoom lenses.
14. The three-dimensional digital entity magnifying glass system as claimed in claim 12, wherein the image chroma-key composite image-processing apparatus varies a degree of transparency of the three-dimensional visual training image of the instructor by a percentage of between 0% and 100% or-changes a color tone of the three-dimensional visual training image.
15. The three-dimensional digital entity magnifying glass system as claimed in claim
14, further comprising:
an image position correction and follow-up apparatus compositely displaying three-dimensionally accurate equal-scaling three-dimensional perspective images into the medical practitioner's visual field image displayed on the right-side LCD and the left-side LCD of the three-dimensional digital entity magnifying glass.
16. The three-dimensional digital entity magnifying glass system as claimed in claim
15, wherein a three-dimensional anatomical image of an arbitrary item is composed
into the image of the medical practitioner's three-dimensional digital entity magnifying
glass and superimposed one on the other, wherein positions of the images are
automatically corrected by the image position correction and follow-up apparatus in
real time.

Documents:

6157-delnp-2006-1-Correspondence Others-(16-12-2013).pdf

6157-DELNP-2006-Abstract-(07-07-2011).pdf

6157-delnp-2006-abstract.pdf

6157-DELNP-2006-Claims-(07-07-2011).pdf

6157-delnp-2006-Claims-(19-05-2014).pdf

6157-delnp-2006-claims.pdf

6157-DELNP-2006-Correspondence Others-(07-07-2011).pdf

6157-delnp-2006-Correspondence Others-(16-12-2013).pdf

6157-delnp-2006-Correspondence Others-(22-04-2014).pdf

6157-delnp-2006-Correspondence Others-(29-04-2014).pdf

6157-delnp-2006-Correspondence-Others-(19-05-2014).pdf

6157-delnp-2006-correspondence-others.pdf

6157-delnp-2006-description (complete).pdf

6157-DELNP-2006-Drawings-(07-07-2011).pdf

6157-delnp-2006-drawings.pdf

6157-delnp-2006-form-1.pdf

6157-delnp-2006-form-18.pdf

6157-delnp-2006-Form-2-(07-07-2011).pdf

6157-delnp-2006-Form-3-(19-05-2014).pdf

6157-delnp-2006-form-5.pdf

6157-delnp-2006-GPA-(29-04-2014).pdf

6157-delnp-2006-gpa.pdf

6157-delnp-2006-pct-search report.pdf

6157-delnp-2006-Petition-137-(19-05-2014).pdf

abstract.jpg

FORM-13[19-04-2014].pdf

POA[19-04-2014].pdf


Patent Number 261066
Indian Patent Application Number 6157/DELNP/2006
PG Journal Number 23/2014
Publication Date 06-Jun-2014
Grant Date 02-Jun-2014
Date of Filing 20-Oct-2006
Name of Patentee TAKAHASHI ATSUSHI
Applicant Address 20-15-1 KIZAKI, TSURUGA VSHI, FUKUI, 9140814, JAPAN
Inventors:
# Inventor's Name Inventor's Address
1 TAKAHASHI ATSUSHI 20-15-1 KIZAKI, TSURUGA VSHI,FUKUI, 9140814, JAPAN
PCT International Classification Number A61B 19/00
PCT International Application Number PCT/JP2005/004758
PCT International Filing date 2005-03-17
PCT Conventions:
# PCT Application Number Date of Convention Priority Country
1 PCT/JP/2005004758 2004-03-17 Japan