In AI-based devices/algorithm-based set-up, the patient’s consent becomes one of the leading challenges. It is challenging to understand the data flow because of unexpected correlations. A simple solution to this challenge is to work only on an anonymized dataset.
Data privacy and cybersecurity challenges also impact the big data/AI based devices. Some of the risks arising may be the result of lack of cybersecurity controls (access controls, encryption, logs), and lack of privacy controls (data minimization isn’t followed, lack of data segregation, unattended data such as individual’s preference capture due to corporate policies etc.)
While data privacy and cybersecurity challenges can be solved by implementing traditional controls, and following the regulatory guidelines, the challenges around reliable & trusted recommendation and algorithm biases are comparatively niche and hence difficult to solve. Automated decisions run “behind the scenes”, thus making it difficult to scrutinize. Thus, to minimize the biased outcomes, one must consider non-technical attributes of devices, in addition to the technical specifications and controls. Missing on either of the aspects may not help device manufacturers to solve this challenge. Other key success factor will be cooperation within industry to push for larger adoption of the AI-algorithm assessment for bias.
To overcome the data privacy and cybersecurity challenge, organizations in the healthcare sector must conduct a Data Privacy Impact Assessment (DPIA), along with assessment of the underlying algorithms for these medical devices, at the beginning of the project.
Wipro’s approach to address privacy challenges
Wipro recommends a risk-based approach to address privacy challenges. Such an approach must begin by asking a key question – why is DPIA needed in the given context? AI-based medical devices (or in general medical devices) collect and use personal data that has an impact on the privacy of individual,if mismanaged.
This becomes the key reason for carrying out a DPIA. Other reasons could be compliance requirements, building trust among stakeholders, and identifying and addressing the risks to the personal data of the patients
An approach to solve these challenges takes 5 steps, with emphasis on how well the data is being stored and managed by medical devices. It should begin by understanding and documenting the inherent features and specifications of medical devices, controls that are in place, and potential vulnerabilities that can be exploited in the light of prevalent threats.