Evaluation of large volumes of published medical literature, in an attempt to understand how specific characteristics have changed over time, has become increasingly common.  Examples include efforts to determine and identify “classic” or highly cited references in subspecialty areas of Orthopaedic Surgery, and assessment of the changing contributions to the Orthopaedic and Traumatology literature originating from specific countries of origin over a period of time [6,7].  In addition, researchers have investigated trends in author gender in high impact journals, as well as attempted to determine trends in the numbers and types of articles published in certain specialty publications over a fixed period [8,9].

Machine learning is a form of artificial intelligence that involves the use of algorithms that give computers the ability to learn without specific human input of rules or instructions that may potentially introduce bias.  Speech and image recognition activities, as well as self-navigating vehicles and facial recognition programs that are used daily, are all examples of the extension of machine learning and artificial intelligence into modern life. Thus far, reports of orthopaedically-related clinical applications of ML include use in the process of scoliosis diagnosis and in the assessment of bone age [10,11].

One of the initial descriptions of ML to analyze a large volume of scientific text was published by Griffiths and Steyvers in 2004 [12]. The authors used these methods to mine the abstracts published in the Proceedings of the National Academy of Science (PNAS) from 1991 to 2001 to extract a set of topics from those abstracts. These topics were then analyzed to determine relationships between scientific disciplines and assess trends in the topics over time.

Sing, et al utilized further adaptations in ML (using the LDAShiny tool) to review orthopaedic subspecialty literature in 2017 [1].  Their goal was to apply available ML computing techniques to assess the Spine-related literature published from 1978-2015.  This allowed the authors to mine a large volume of text from abstracts in an unbiased fashion to discover thematic topics and identify 100 research topics within the spine literature, and then to determine “hot” and “cold” trends during that time period.  The authors felt that the ability to examine large volumes of text, while minimizing the risk of bias or human error, could lead to improvements in the processes of determining and updating clinical guidelines, as well as provide direction for future research. They recognized that the process of labeling the topics, or associating them with specific clinical situations or activities, did require further input from those with specialized knowledge [1].

While a detailed description of the computer science and mathematics upon which these techniques are based is beyond the scope of this current report, it is important to highlight and describe the concept of the “Topic” which is the fundamental finding reported by this type of analysis. As noted in the Materials and Methods section of this paper, thematic topics can be thought of as combinations of words that have a high tendency or likelihood of appearing together within a segment of text.  These topics can then be analyzed for increasing or decreasing popularity over time, and thus topic trends can be viewed over time. Finally, the abstract/paper that matches the topic in question most closely can be identified through a best-fitting process. 

We used a readily available form of machine learning (LDAShiny) to analyze text in a manner similar to that reported by Sing, et al, and focused specifically on data published over a twenty-year period in two journals with an exclusive focus on pediatric orthopaedics [1]. Correlation of the topic trend data, associated publication numbers, and the best-fit articles generated some interesting findings and further questions.  It is not a surprise that 2 of the “hottest” topics (topics 31, 48) also fall in the top 6 topics with greatest overall prevalence and number of associated abstracts during the period of study.  However, it is interesting that one of the top 10 most published topics, with a best fit abstract title focused on gait in cerebral palsy patients, was noted to fall in the “Coldest” category (topic 10).  Review of the graph in Figure 2 illustrates a possible explanation, in that it appears that there was a marked peak in that topic early on in the study period, and then a precipitous drop-off, or negative slope, over time. This specific finding may suggest that there is an opportunity for more detailed analysis of sources of literature that investigates orthopaedic care for patients with cerebral palsy.

Another interesting finding includes the trend data of topic 23, which correlated most closely with an abstract regarding surgical complications, and was strongly positive to the point of reaching the top 3 “Hottest” category.  However, the overall number of associated abstracts over time was not sufficient to warrant a position in the overall top ten listing by prevalence.  Finally, the fact that a topic (topic 31) appearing to have little relation to direct patient care or management was found to have the greatest number of associated abstracts, and was amongst the “Hottest” of the topics of the study, seems deserving of further investigation.

As noted by other authors [1], the topics generated can be labeled, or associated, with specific clinical issues. For example, topic 27 is clearly associated with surgical management of adolescent idiopathic scoliosis, while topic 32 involves studies that investigate the measurement of patient outcome data in orthopaedic surgery. The best-fit article, as determined by LDA, for each topic can be a helpful guide, but in many cases this decision-making requires specialized knowledge of the subject matter being analyzed. However, correlation of these topics to a specific clinical situation or label requires input from those within the specialty field, which can be affected by bias or error.

There are potential limitations to this study.  One of the most obvious is the relative lack of familiarity and facility of most pediatric orthopaedic clinicians (the primary audience for the journals reviewed) with the computing methods and technologies employed in generating the data used.  The authors hope that they have clarified the terminology (such as topics) sufficiently.  In addition, there is the concern expressed by Sing, et al [1] that one of the limitations of this type of an approach to the literature is that it is not possible to factor in any concerns or issues with the quality of the journals or publications undergoing analysis.  While that may be an issue in some fields, we do not share a similar concern regarding the sources of the abstracts reviewed for this investigation due to the reputation of the source journals within the field of Pediatric Orthopaedics.  We acknowledge that many articles regarding Pediatric Orthopaedics are published in journals other than JPO A&B. However, the authors felt that the process of manually identifying articles from non-Pediatric Orthopaedic-specific sources by title could have been an unnecessary source of bias or error.  As such, the literature sources were limited to review of JPO A&B in an attempt to minimize that potential issue.

Finally, despite the fact that we actively edited and added multiple “Stop Words” (Table 1) that were beyond those chosen by the program itself, it may be that we should have been more liberal with word elimination.  Whether greater selectivity would have pushed the topics toward a point at which they were more recognizable or more easily associated by clinicians with obviously clinical situations will require further evaluation.

Overall, these results are particularly interesting, and could prove to be a valuable starting point for the use of these methods to analyze the Pediatric Orthopaedic literature over periods of time.  It is a very different way to look at the literature, but the use of an algorithmic-based program has the potential benefits of limiting human bias or error in the process.  We concur with Griffiths and Steyvers [12] that the ability to determine topics that are either “hot” or “cold”, and the ability to do it in an objective fashion, is one of the most important applications of these processes.  Our analysis, specifically the trending of the topics, raises questions that may warrant further investigation.

We believe that this effort is important in that it demonstrates what is currently technologically possible, and hints at the level of information that could be determined through use of these rapidly evolving techniques in the future.  The advantages of generating and using this type of information, which should minimize the potential for human error or bias, is obvious. We agree with Sing, et al [1] that these techniques could improve the processes of formation and updating of clinical practice guidelines, as well as help guide the directions of future research. Finally, Bashir et al reported that the majority of systematic reviews that have been published are not updated to reflect new evidence [13]. Machine learning methods, similar to those used in this demonstration, could be utilized to analyze and re-analyze large amounts of published data to keep up with new studies and ensure that recommendations and clinical care guidelines reflect all available data.