Ⅰ. INTRODUCTION

Osteoporosis is defined as a systemic skeletal disease featuring the reduction of bone mass and abnormal ultrastructure in which bones are weakened as a result and are prone to fracture^1,2). Osteoporosis is diagnosed through bone density measurement and is diagnosed when bone density is below the T score of -2.5 based on the average bone density of adults³⁾. Osteoporosis eventually causes bone fractures, and these fractures reduce the quality of patients’ lives. Furthermore, it may lead to early death. In addition, most osteoporosis patients have no symptoms, and because one-third of patients with osteoporosis-induced vertebral fractures have no pain, they do not receive treatment, leading to secondary fractures or the increased possibility of death⁴⁾. Moreover, early diagnosis of osteoporosis is difficult because patients have no symptoms^{5, 6)}.

Various equipment and methods are used to measure bone density. Dual energy X-ray absorptiometry(DEXA) or quantitative computed tomography(QCT), quantitative ultrasound( QUS), etc. is being used. Among these, DEXA has high accuracy and is a standard method that can apply WHO’s osteoporosis diagnostic criteria. However, these types of bone density measurements are complicated and costly and the usage is limited⁷⁾. Meanwhile, it is easy to diagnose osteoporosis with panoramic radiographs that are taken routinly for a dental examination. In cases of osteoporosis diagnosed using panoramic radiographs, there are studies that have reported having up to 100% accuracy, implying a very high correlation with bone density test results ^4,8,9). Diagnostic methods for diagnosing osteoporosis on panoramic radiographs include mandibular cortical index( MCI), mandibular cortical width(MCW), panoramic mandibular index(PMI), alveolar crest resorption degree ratio, and fractal dimension^9-12). Among these, diagnostic methods using the mandibular cortical index(MCI) and mandibular cortical width(MCW) have a high correlation with the bone density test using DEXA⁸⁾. Therefore, panoramic radiographs, which can be used easier to most patients, are relatively inexpensive, and have a high correlation with bone density test results, is useful for large-scale screening of osteoporosis¹⁴⁾.

A study was reported about the ability of artificial intelligence( AI) to determine osteoporosis in the panoramic radiograph. Among them, it is largely divided into the fuzzy neural network(FNN) and convolutional neural network( CNN). The ability of AI to determine osteoporosis varies depending on the research conditions. It was reported to be very high in cases of using FNN, with 76.2~100% sensitivity, 63.2~99.8% specification, 71.3~98.9% accuracy, and in cases of using CNN, it had 91.5~100% sensitivity, 52.9~99.0% specification, 84.84~99.25% accuracy^15-22). CNN generally consists of a convolutional layer and a pooling layer. The convolutional layer, as a feature extractor, comprises the neural network with a feature map extracted from exploring and training the same feature parts of the input images. The pooling layer lowers the feature map’s resolution enabling it to perceive the object without being affected by the object’s location or distortion. By using maximum pooling, speed increases, and the discernment of image increases. The fully connected layer presents the convolutional layer and the pooling layer in abstract information and reduces training errors through backpropagation ²³⁾. CNN is learned by supervised learning. It composes the neural network by automatically extracting the features of images that have been input during the training process and through this, provides the result by classifying data²⁴⁾. Lately, in the ImageNet large scale visual recognition challenge, CNN showed a smaller error rate than that of humans in classifying images²⁵⁾.

Osteoporosis affects the entire skeletal system and progresses faster in the cancellous bone than in the cortical bone²¹⁾. Because the diagnosis of osteoporosis in the panoramic radiograph is interpreted by verifying the lower mandible cortical bone, in studies applying AI, training and testing were conducted limited to the rectangular region of interest(ROI) including the lower mandible of the cortical bone. Sensitive areas were reported to be not only cortical bones but also areas including cancellous bone^20,21). The study about the ability to determine osteoporosis of CNN using panoramic radiographs presented so far has been conducted with limited ROI to local areas including the cortical bone of the mandible and was not studied for the entire panoramic radiographs.

Therefore, in this study, the purpose is to use the entire images of panoramic radiographs to verify the ability of CNN in determining osteoporosis, rather than limiting ROI, and to verify the area where it reacts sensitively.

Ⅱ. Materials and Methods

1. Materials

This study has been approved by the Chonnam National University Dental Hospital(CNUDH) Institutional Review Board(CNUDH-2017-014)²¹⁾. Panoramic radiographs from a total of 1,156(mean age 49.0±24.0 yrs) patients were used for this study and among them are 537(mean age 73.7±7.8 yrs) osteoporosis patients, and 619(mean age 27.5±5.4 yrs) normal patients(Table 1, 2). The panoramic radiographs have been obtained from patients who visited CNUDH from 2008 to 2017 and were diagnosed as either osteoporosis or the normal by the agreement by two Oral and Maxillofacial Radiology specialists on the basis of interpreting the shape mandible inferior cortex of the panoramic radiographs, in accordance with the classification of Klemetti et al^12,21). The panoramic radiographs were obtained with Kodak 8000C digital panoramic equipment (Carestream Health Inc., Rochester, NY) in the parameter of 71 kVp, 12 mA, and 13.2 s.

2. Deep learning model

The panoramic radiographs were reduced in the size from the varying pixel values from 2424×1244 to 3036×1536 to a resolution of 312×312. Augmentation of the panoramic radiogrpahs was performed to apply deep learning algorithms( Figure 1). VGG16 deep learning CNN model was used for this study. The VGG16 is connected with five CNN layer groups and two fully connected networks(FCN) and is ultimately determined through the softmax classifier layer. In this study, the global average pooling(GAP) layer was used instead of the FCN for the application of gradient- weighted class activation mapping(Grad-CAM). The stochastic gradient descent(SGD) was used as the optimizer for training, and the training rate(0.005), and momentum( 0.9) was adjusted. The categorical cross entropy was used for the loss function, and the batch size for both training and validation data was 16, and 150 epochs were performed. To interpret the decision-making process by the deep learning model, heat map images through Grad-CAM visualization were formed to visualize the areas that had a major influence in determining osteoporosis on each radiograph. The heat map image of the VGG16 deep learning model presented a distribution of colors from blue(low) to red(high) depending on the degree of how sensitively CNN reacted to determine osteoporosis and the normal.

Ⅲ. Results

Among the panoramic radiographs of 619 normal groups(mean age 27.5±5.4 yrs) and 537 osteoporosis groups(mean age 73.7±7.8 yrs), the panoramic radiographs of 433 normal groups(mean age 27.7±5.4 yrs) and 383 osteoporosis groups(mean age 74.0±7.6 yrs) were used for the study of CNN, and 93 normal groups(mean age 26.4±5.1 yrs) and 72 osteoporosis groups(mean age 73.7±8.0 yrs) were used for testing. As a result, The determination of CNN’s osteoporosis for the entire area of panoramic radiographs was shown with 100% accuracy(Table 3).

In a total of 165 panoramic radiographs that had been diagnosed as osteoporosis and the normal by CNN, the area that had an influence on determining osteoporosis and the normal in the panoramic radiographs was visualized through Grad-CAM depending on the degree of influence by representing the distribution of colors(Figure 2, 3).

In heat map images through Grad-CAM visualization of the 72 panoramic radiographs with osteoporosis, areas that were colored red were cranial base in 30.6%(22/72), mandible inferior cortex in 72.2%(52/72), the inferior area of the mandible inferior border in 43.1%(31/72), the cervical vertebrae in 70.8%(51/72), the cancellous bone of the mandible in 33.3%(24/72), the cancellous bone of the maxilla in 20.8%(15/72), the dental area in 6.1%(5/72)(Table 4). In the heat map images of the 93 panoramic radiographs of the normal, areas that were colored red were in dental area in 98.9%(92/93), the mandible inferior cortex in 4.3%(4/93), the cancellous bone of the mandible in 7.5%(7/93), the cancellous bone of the maxilla in 5.4%(5/93)(Table 5).

Ⅳ. Discussion

This study aimed to verifying the areas reacting sensitively and verifying the accuracy of CNN’s osteoporosis determination by using the entire image instead of setting the limited area as ROI on panoramic radiographs.

Osteoporosis, which can lead to fractures and death, is fatal to the quality of life of the elderly, therefore early diagnosis, as well as prevention, is important. In our country, insurance coverage for medication is available in cases of those who are diagnosed with osteoporosis in clinical trials and are below the DEXA standard T score of -3.0 published by the WHO⁶⁾.

According to a domestic study about the prevalence rate of osteoporosis based on the National Health Insurance Service data, 1,564,091 people received treatment for osteoporosis in 2008, and among them, there were 171,902 males and 1,392,189 females[26]. A fracture in osteoporosis is defined as a ‘fracture caused by the degree of external force when the patient falls from a standing position’. Among fractures in adults over the age of 50, it mostly occurred in the vertebrae, femur, wrist and humerus, and additionally occurs in pelvis, sacrum, ribs and ankles²⁷⁾. Among them, especially the fracture of femur caused a higher possibility of limiting the movements of patients and leading to early death compared to other fractures. It has been published that, compared to the death rate of 7.0% after 1 year of the fracture of the vertebrae, the death rate after 1 year of the fracture of the femur was 16.7%, and that our country has shown the highest increase in the fracture rate of the femur compared to other countries^28-32). In a study on the prediction of the fracture of the femur considering the current elderly population growth, from 2016 to 2025, it was predicted that patients with femur fracture would increase from 35,729 to 52,358³²⁾.

The use of AI is increasing in recent various diagnostic imaging fields. Previous studies reported CNN’s interpretatibility of osteoporosis was verified as 91.5~100% sensitivity, 52.9~99.0% specificity, and 84.84~99.5% accuracy^15-17,19,20). The results of the interpretibility of AI for osteoporosis may depends on the conditions of the subject of the experiment and ROI setting. One of the difficulties in the study of osteoporosis diagnosis ability on panoramic radiographs using AI is how ROI should be set. While Oral and Maxillofacial Radiology specialists determine the diagnosis of osteoporosis by limiting to the mandible inferior cortex on panoramic radiographs, and when on the same conditions, trying to input in the AI by only extracting the mandible inferior cortex, there is a limit that the data processing is difficult. Therefore, ROI that was set in rectangular shapes including the mandible inferior cortex was used in previous studies^19-20).

In this study, CNN reacted sensitively to the cancellous bone, which might be because CNN has a higher perceptibility than human eyes. Human eyes cannot diagnose osteoporosis through the cancellous bone on panoramic radiographs because there is a limit in distinguishing the grayscale. While radiographs and computed tomography( CT) have up to 12~16 bit(4,096~65,536 levels of grayscale), medical monitors generally have a level of grayscale only from 256 to 1,024, and although monitors that demonstrate more grayscales have been developed, human eyes cannot distinguish grayscale above 900 levels[33]. Since CNN can distinguish a wider range of grayscales, CNN could extract the features of cancellous bone which humans could not distinguish; it will be a valuable information for determining the early progression of osteoporosis²¹⁾. From the sensitive reaction CNN has in the cancellous bone, it can be understood that there was a greater change in the cancellous bone than the cortical bone in osteoporosis. Therefore, it can be assumed that AI has no limit in sensitivity to limited areas that humans see. Moreover, the fact that osteoporosis is not a localized disease that is confined to the mandible inferior cortex, but is a generalized disease that occurs in the whole skeleton, makes us remind that the AI can sense the changes in the entire bone included on panoramic radiographs.

In this study, the research subjects with statistically significant differences in age groups between the osteoporosis group and the normal group were used, rather than to confirm accuracy, but to clearly see where the most sensitive area is when diagnosing osteoporosis(Table 2). When inputting the entire area of panoramic radiographs and not limiting ROI, the AI showed sensitive areas not only for the mandible inferior cortex(72.2%), but to the cervical vertebrae( 70.8%), the cancellous bone of the mandible(33.3%), cancellous bone of the maxilla(20.8%), cranial base(30.6%) as well. From the changes in bones occurring by osteoporosis, it is shown that the AI reacts sensitively in various bones. Various devices and methods are used to measure bone density. Among them, the vertebrae and femur are examined by mainly DEXA that has high accuracy and precision ⁶⁾. The sensitive area of AI was also high in the cervical vertebrae than the mandible infereior cortex that is used in osteoporosis diagnosis using panoramic radiographs, like in DEXA. Bone changes in the vertebrae shown on panoramic radiographs were detected. Similarly, in the study about osteoporosis diagnosis using cone beam computed tomography(CBCT), when comparing the bone density of the cervical vertebrae measured using CBCT and the bone density of lumbar vertebrae and femur measured using DEXA, it showed a fairly high correlation of sensitivity which was over 70%[34]. Meanwhile, in cases of the AI determining as the normal, the sensitive area was low in mandible inferior cortex with 4.3%(4/93), the cancellous bone of the mandible with 7.5%(7/93), and the cancellous bone of the maxilla with 5.4%(5/93), but the dental area was shown fairly high with 98.9%(92/93)(Fig 2-3, Table 4-5).

The limit of this study was that the age difference of the study subjects was high and that the number of the subjects was small(Table 1-2). However, this study showed that the AI had a very high ability in osteoporosis diagnosis and that it can detect bone changes in various bones including vertebrae( 70.8%). Henceforth, a study on diagnosing osteoporosis without limiting the area of panoramic radiographs in AI is needed in the near future with increased number of study subjects and decreased age difference. In addition, when the AI determines the normal on panoramic radiographs, the dental areas were found to be sensitive areas in most images(98.9%%). A study excluding the influence of dental areas might be needed to increase the quality of the study.