On November 1st, Nanjing-Tencent AI Lab announced today that its AI+ medical field research has taken the lead in image analysis from the screening of AI+ medical fields. Relevant "smart microscope> The project is already in the R&D testing phase.
The intelligent microscope incorporates artificial intelligence (AI) vision, speech, natural language processing technology, and augmented reality (AR) technology. Doctors can easily input voice commands, and AI can automatically identify, detect, quantify, and generate reports. The results are displayed in real time to the eyepieces that the doctors look at, promptly reminding and not interrupting the doctor's reading process, which can improve the diagnostic efficiency and accuracy of the doctor.
The general diagnostic process includes four major steps: medical imaging screening, pathological analysis, planned treatment, and postoperative rehabilitation and tracking. Tencent's goal is to create an AI product that covers the entire process of diagnosis and treatment. In medical imaging screening, “Tencent 觅影” has made gratifying progress. Currently, there are more than 100 top three hospitals in the country, and the number of auxiliary doctors reading more than 100 million images, serving over one million patients, suggesting 150,000 high-risk lesions. . Now entering the pathology analysis phase provides a solid foundation for the company to build a full-stack AI+ medical solution.
The following is a description of pathological AI technology, “smart microscope”, research project background, and “Tencent 觅影””, the gratifying progress in medical imaging, shared by Dr. Yao Jianhua, medical expert of Tencent AI Lab AI+, and Dr. Han Wei, AI+ pathologist . Due to space requirements, the full text is abridged.
Part I: Pathology AI Analysis and Smart Microscopy
1. Introduction to common pathological diagnosis tools
The main method of doctor's pathological diagnosis is to observe the section, magnify it 40 to 400 times, and observe the cell morphology and tissue structure for diagnosis. Pathology and digital pathology scanners are the most commonly used tools for doctors.
The microscope has a history of more than three hundred years, and the doctor can use it skillfully and the price is low. However, its limitations are as follows: First, the field of view is small. The doctor can only see a small part of the slice from the eyepiece at a time. It is necessary to switch multiple fields of view and associate them to get the overall diagnosis. Second, the image is not digitized. Read with the AI algorithm.
2. Introduction to intelligent microscope and test function
Smart microscopes break through the limitations of traditional microscopes and were previously passively used, now turning to active assistive physicians, such as computer vision to help doctors, from simple but cumbersome cell metrology to difficult and complex cancer type identification and precise regional division. At the same time, the use of speech recognition allows doctors and intelligent microscopes to perform smooth human-computer interaction. Finally, the final pathology report generation is assisted by natural language processing techniques.
When reading a film, the doctor only needs to give a voice command, the AI can automatically read the film, automatically collect the image, and assist the doctor to diagnose; after the doctor completes the film, give the “Generate Report” instruction, the microscope can microscope Screenshots and diagnostic results are populated into report templates, reports are automatically generated, and doctors review the results and publish reports, making the most costly report generation quick and worry-free.
Intelligent microscope technology module introduction
3, pathology AI analysis is the future research direction
Pathological analysis is the gold standard for diagnosis, prognostic analysis, and guiding cancer treatment. In China, about 120,000 pathologists are needed in 2017, but there are fewer than 20,000 trained pathologists. This gap is still increasing year by year, so the pathologists are in short supply and heavy tasks. Digitizing the pathological sections and assisting the analysis with the AI algorithm helps to alleviate the shortage of pathologists and is the future trend of AI+ medical care. In addition, the study of basic pathological AI can promote the ability of pathological AI in three aspects:
1. AI-based pathological diagnosis model: can improve the diagnostic efficiency and improve the ability to identify small lesions and difficult cases;
2. AI-based pathological prognostic prediction model: such as predicting five-year overall survival rate, five-year disease-free survival rate, and five-year no-distance metastasis survival rate;
3. Pathological omics: extract features from pathological data, quantitatively analyze and explore the correlation between pathological features and diagnosis and treatment.
The first type allows doctors to “do more and better”, and the latter two allow doctors to “break through the fog” to break through the difficulties.
Demonstration 1: mitotic cell detection
The mitotic cell count measures the activity of cancer cells and is an important indicator of cancer diagnosis and grading.
Under the traditional microscope, this is a very cumbersome process — & doctor; to observe 10 different areas under high magnification, accurately identify mitotic cells, and then count the number. Under the intelligent microscope, when the doctor moves to the target area, he only needs to give “Silk Split”, a simple voice command. The AI algorithm can automatically identify, detect and count the results of the area, and immediately display it to the eyepiece field viewed by the doctor. . Once an area is complete, the doctor can move to a new field of view, repeat the voice command, and the AI will update the results.
Demonstration 2: Quantitative analysis of immunohistochemistry —— taking Ki-67 stained sections as an example
Immunohistochemistry is the application of antigen-antibody binding. The specific staining of specific proteins in tissues can be performed on pathological sections to provide more accurate cancer diagnosis at the molecular or genetic level.
Taking Ki-67 stained sections as an example, it can help determine the proliferation index of cancer cells. Under a traditional microscope, the pathologist needs to count the number of cancer cells stained brown (positive) and the proportion of cancer cells on the slices. Under the intelligent microscope, the doctor only needs to give the voice command “Ki-67”, and the AI automatically completes the cell counting and the ratio calculation.
Demonstration 3: Cancer Regional Monitoring & mdash;— taking lymph node section as an example
Lymph nodes are a common pathway for cancer spread or metastasis. The doctor can check the lymph nodes near the cancer area to confirm the cancer stage (TNM stage) and then decide on the treatment plan.
Under a traditional microscope, when using a low power microscope, doctors tend to miss small cancer areas, and using a high power microscope is very time consuming. Under the intelligent microscope, the doctor only needs to give the “area detection” voice command. The AI will identify the suspected cancer area, accurately estimate the size of the area, and display the results in real time to the eyepiece view of the doctor to avoid omissions.
Demo 4: Auxiliary Diagnostic Process & mdash;— Taking Polyp Classification as an Example
The intelligent microscope also assists the doctor in completing an overall diagnostic process and automatically generates a diagnostic report.
In the case of colorectal screening, polyps are generally found to be pathologically diagnosed to determine their benign and malignant and specific lesion categories (adenomas, adenocarcinomas, etc.). If it is benign, only follow-up is required, and malignancy requires surgery in time.
When using manual diagnostic screening, there may be large differences between experienced and young doctors, and the results are subjective. And AI trained by a large amount of expert data can improve the consistency of diagnosis. Under the intelligent microscope, the doctor only needs to give the “polyp classification” voice command, AI can automatically read the film, automatically collect images, and assist the doctor to diagnose. After the doctor's reading is completed, the “Generate Report” instruction is given, and the AI automatically generates a report for the doctor to review the results and release the report.
The second part: "Tencent Shadow" & medical image screening case
The general diagnostic process includes four major steps: medical imaging screening, pathological analysis, planned treatment, and postoperative rehabilitation and tracking. In the field of “medical imaging screening”, Tencent’s AI medical solutions experts ——“Tencent 觅影” have made good progress. It uses AI medical imaging analysis to assist doctors in screening for esophageal cancer, pulmonary nodules, diabetic retinopathy, colorectal cancer, breast cancer, cervical cancer, etc., and to use AI's auxiliary engine to assist doctors in identifying more than 700 disease risks. prediction.
Tencent Medical AI product landing situation
Tencent Yingying has reached cooperation with more than 100 top three hospitals in China to promote the research and application of AI in the medical field through the joint construction of artificial intelligence combined medical laboratory. As of October 2018, it has been launched in more than 100 top three hospitals nationwide, and has assisted doctors to read more than 100 million medical images, serving more than one million patients, suggesting 150,000 high-risk lesions. At the same time, in the case of assisting doctors in case analysis, in terms of a single hospital, it has assisted doctors to analyze 7 million outpatient cases, suggesting a high risk of 170,000 times, effectively assisting clinicians to improve diagnostic accuracy and efficiency.
1, AI pre-diagnosis auxiliary diagnosis system
This system first uses medical books and past medical records to create medical knowledge maps, builds knowledge reasoning models and deep learning models, connects patient medical history and disease with knowledge maps, and draws possible disease judgments, and gives structured diagnosis reports and recommendations. Treatment programs. At present, the pre-diagnosis-assisted diagnosis system can cover about 700 common diseases (WHO30000 diseases), and this auxiliary system gives the possibility of suffering certain diseases according to the input of the medical history. The accuracy of the existing system can reach 93% of Top3 diseases and 96% of Top5 diseases.
2, medical imaging case - colorectal cancer screening test
Colorectal cancer is one of the five major malignant tumors, with more than 1.2 million new cases each year, and the incidence is increasing year by year. As with most cancers, early diagnosis and treatment is critical. According to statistics, the 5-year survival rate of early colorectal cancer can reach 90%, and the local progression can reach 70%, but it is less than 10% in the late stage. Therefore, the World Health Organization recommends that people who are 50 years old should be screened every five years. The main test for colorectal cancer is the digestive tract endoscope.
Colorectal cancer screening
The algorithm of Tencent AI Lab is based on deep learning, which divides the image into small pieces, calculates the possibility of polyps on each block, and then combines them to locate polyps. A classifier is also trained to distinguish between adenocarcinoma and adenoma. The AI algorithm enables real-time video stream diagnostics during the doctor's exam. In clinical use, when the doctor is using the endoscope, the AI algorithm detects polyps in the background and reminds the doctor. The technical feature is that the result is real-time, which requires fast speed and high requirements on the algorithm. Among them, the accuracy of polyp positioning can reach 96.93%, and the differentiation of adenocarcinoma is 97.2%.
Colorectal cancer screening algorithm
3, medical imaging case - early screening for breast cancer
Breast cancer is the most common malignant tumor in women, and its incidence is increasing year by year. This trend is more serious in China. In 2012, the incidence of breast cancer in China only accounted for 11.2% of the world's total, and will increase to 29.8% by 2030. One of the main methods of breast cancer detection is mammography. This is a method of detecting X-rays that is simple and non-invasive to the patient. Because of the X-ray projection, the tissues overlap each other, and it takes a lot of experience to distinguish between normal glands and suspected lesions on the molybdenum target.
The algorithm framework of Tencent AI Lab is: a multi-window deep learning network that simultaneously inputs the CC and MLO projections of the left and right breasts into the network. Three different models were trained to perform lump testing, calcification testing, and benign and malignant judgments. Difficult mining and algorithmic iterative processes have also been designed to improve performance. The doctor examines the algorithm results and feeds them back to the model for further optimization. In the latest product, the sensitivity of detecting calcification is 99%, the sensitivity of malignant mass is 90%, the sensitivity of benign and malignant and specificity are 87% and 96%, respectively, under 0.2 false positives. The level of the doctor.