Hello and welcome to Careviser by Marie Loubiere. Years of research summarized in 5-minute emails. Spot emerging opportunities in healthcare.
With the launch of the covid-19 vaccines, everybody has become familiar with the terms post-market surveillance and adverse events. The monitoring of the safety of a product (whether pharma or medical device) is mandatory after it’s put on the market. Let’s have a look at how social media and the new European medical regulation (MDR) have changed the game.
Gartland, A., Bate, A., Painter, J.L. et al. Developing Crowdsourced Training Data Sets for Pharmacovigilance Intelligent Automation. Drug Saf 44, 373–382 (2021)
🗝️ Why it matters: Post-marketing safety surveillance activities are time-consuming. They seem to become even busier as there is an increasing number of individual case safety reports. These are crucial activities as not only are they mandatory, but also they can be a matter of life or death.
🔎 The study: Looking at other industries such as the financial sector where the monitoring of some transactions is automated, the authors aim to see it would be possible to automate some elements of the individual case safety reports intake so that there would only be a human review of the medical components of a report. They explored whether machine learning technologies could do that with the following challenges:
Individual case safety reports change frequently as they evolve for different meds and with the state of science
Training the algorithms require large sets of data that are not readily available. Individual case safety reports are in public domain repositories (such as the one for the FDA or the EMA), and are hard to share as clinical data cannot be easily anonymized. Social media data could be used as a proxy but there is an extremely important volume of data on social data related to medicine. It would take tens of thousands of hours to annotate them. That’s how the authors came up with the idea of exploring crowdsourcing the annotation of social media posts related to adverse events. So they built a study to see if crowdsourcing could be effective to build training data sets for automation algorithms.
✅ Findings:
The study started with the evaluation of social media posts on Twitter and Facebook related to 15 products (both prescription and OTC drugs) over a year.
First, they identified 212,000 posts. Then they removed posts that weren’t in English and duplicates. Then they tried to filter out spam or ads but there were still 81,300 posts to review which was way too many for a team of physicians and scientists.
They randomly selected 19% of posts (still 15,490 of them!) to review.
They identified 22 relevant topics that they wanted to extract from a post (identity of the poster either a provider or patient or family member, social-economic indicators, contributing health factors, drug indication, adverse events, efficacy).
It took about a minute to review each post so the team of 13 curators took 5 months to assess all of them.
Then they moved on to the crowdsourcing phase of the study using Amazon’s MTurk where freelancers (called “Turkers”) work on small gigs online.
The researchers asked Turkers to complete a form for each post in which they had to fill out the 22 relevant topics when applicable for the post. They were different phases in the study. In phase 1, three turkers had to review the same post. In phase 2, only one had to review a post but with a higher reward. Phase 1 accuracy was almost 93% and phase 2 accuracy was only down one percentage point at 92%.
Crowdsourcing proved to be an efficient and accurate method. Based on phase 2, it takes 5% of the time of the initial review team to assess all posts.
🚀 Opportunities ahead:
There are some ethical and practical considerations to be solved before crowdsourcing could become an official tool for the training of post-market surveillance automation algorithms, ie. it could be considered as a sweatshop, Turkers could easily find the identity of a poster etc. But it is exciting to see that social listening which is huge in consumer goods could become a thing in healthcare too.
As the new Medical Device Regulation (MDR) took effect this year in the EU, post-market surveillance activities have become well-defined and more stringent. They have to be formalized in a post-market surveillance plan, and for medical devices class II and higher they need to prepare a periodic safety update report based on all the collected data. Manufacturers now have to implement actions that are both not only reactive but also proactive. They need to collect more data to ensure the safety and reliability of their devices beyond pre-market activities and have to notify adverse events faster.
🤯 The problem: Talosix is a US start-up building on that regulatory change to sell a real-world data collection software solution.
🤗 The solution: Talosix collects real-world data to feed three use cases for medical device manufacturers:
Proactive post-market surveillance with the new MDR which they integrate with the existing quality management system of their clients
Reimbursement coverage by CMS for new procedures
Market access to show prospects that a device offers superior efficacy
📈 The traction: Founded in 2018 by a team of analytics executives, seem to be self-funded by the founder. Some members of the founding team left this year. I thought it’d be interesting to see how they pivoted their existing real-world data collection solution to different needs (regulatory and business development).
When I searched for tech companies that use social listening in healthcare, I initially only found data marketing agencies such as Visfo and Convosphere. These did not cater to regulatory use cases. The only company I identified in that field is 8-years-old MedAware Systems. They combine data from literature repositories, adverse event databases and social media. They have a small team and last raised money in 2016. Anyone else interested in entering that space? :)
That’s a wrap for today! Don’t hesitate to reply to this email with comments, I read and answer all emails :)