Skip to content

How Modern Computer Vision Can Revolutionise Drug Discovery


Oct 12, 2021 - 7 minutes read

1995 Blog Post Computer Vision 416X300
Marcel Falkiewicz Leader of Data Science
See all Marcel's posts
Retail Report Resources


Drug discovery is a multi-stage, complex and extremely expensive process. It takes approximately 12-15 years for a new drug from the time it’s discovered to reach market availability, and it can cost more than a billion dollars. It’s no surprise that pharmaceutical companies are looking to shorten the drug discovery process and reduce its cost. Machine learning is a very promising tool that can help in that area.

Computers have always been very proficient at processing structured tabular data. However, processing of unstructured data, such as images, sounds and free text, was much more challenging. Deep neural networks have revolutionised the ability of computers to understand unstructured data, including language, speech, images and other categories. Specific architectures of neural networks have already outperformed traditional approaches based on human-invented algorithm design. The accuracy of object detection by convolutional neural networks has already surpassed humans. In addition to the capability of such networks to index massive amounts of images and thus making them searchable by text, they provide a powerful tool that can help scientists discover drugs quicker.

Early Stage of Drug Discovery

In this part, we’re going to focus on a specific process of early stage drug discovery during which the scientists are testing candidate drugs on artificial cell cultures. The cells are grown on microplates that are composed of tens or hundreds wells. Each well with cells is treated with a specific dose of the candidate compound. The scientists are interested in the compound’s mode of action (MOA). For example, they might be interested whether the compound is internalised by the cell nucleus. In order to check that, they‘re marking specific cell compartments with fluorescent dyes. This allows to visualise the cells under a special microscope. With the latest high-content screening (HCS) microscopes, a single experiment can yield images of millions of cells. A single procedure that might involve tens or even hundreds of plates is called an “assay”. It’s simply impossible for a human to go through all the results.

The Traditional Approach to Computer Vision

Computer vision tools that were developed before the deep learning revolution offered a remedy to the data volume problem. The scientists were equipped with a set of algorithms that they could use to process the data. You can think of these algorithms as tools that can be used to process the raw material (microscope images). If the scientists were able to find the right tools and apply them in a correct order, they could isolate a specific feature that distinguished the cells affected by the drug from those not affected. If everything worked, the rest was only a matter of counting the affected and non-affected cells. However, there are several problems associated with the classical approach to computer vision.

Firstly, every experiment needs a pipeline designed specifically to handle it. Building pipelines is a time-consuming process and requires substantial expertise. Secondly, when a pipeline is designed, the scientist is usually only able to visually inspect its effectiveness on a small subset of data. As a result, they have to assume that it remains consistent for the unseen data. Sometimes it leads to decreased veracity of the experiment due to inclusion of contaminated wells from the unseen data subset and incorrect results from others. Thirdly, by design, the data is transferred to the machine where the pipeline is running, usually the scientist’s PC or a remote server. It’s saved in a folder and not available to other scientists unless specific data governance procedures are developed. This makes sharing of the results more challenging. Finally, such tools are usually not suitable for massively parallel processing. This results in very long processing times that can fail unexpectedly due to lack of resources, such as memory or disk space.

Computer vision tools based on deep neural networks can address nearly all the above-mentioned problems. This statement can come as a surprise, especially considering the expertise required to build and train neural networks. However, neural networks already are an integral part of our everyday lives — they attempt to show you relevant ads on your favourite social network apps and in every online shop, they filter content that’s recommended to you on your favourite streaming services and identify your friends on the pictures that you take, just to name a few examples. Yet, you might not be aware of that, because. These tools can be integrated into scientific data analytics in exactly the same way — hidden behind the scenes and providing only the outputs that are relevant for the scientists and their research. This is how the revolution should take place.

Scientists and Computers Working Together

The shift towards processing data using deep neural networks also provides a very deep change in the relationship between the scientist and the software they’re using to analyse data. In the traditional approach, the scientist was equipped with a set of tools and had to leverage them in an optimal way. With the neural-network-based approach, the scientist receives an assistant that’s capable of ingesting the relevant knowledge and using it to process massive amounts of data. A properly designed user interface can then present the results to the scientist in a way that maximises productivity and minimises risk of mistakes at the same time. But how can this be achieved?

In the simplest terms, you can think of a neural network for computer vision as a two-step process. At first, it turns images into a set of numbers that’s smaller than the number of pixels in the image (otherwise it could just take all the pixels). Each of those numbers should encode a single feature of the image — for example, the roundness of the cell or the brightness of the cell membrane. Then, these numbers are combined to make a prediction about the cell class or phenotype. The key feature of the neural network is that it learns to do both of the above tasks from the data. In the right circumstances, the network is able to generalise to new, unseen data and process it accordingly.

This is very unlike the traditional data processing pipelines that the scientists used to build. Contrary to the fixed nature of the traditional pipelines that require building a different processing pipeline for each dataset, a neural network provides flexibility that allows it to adapt to a wide range of datasets. The scientist doesn’t have to know anything about the network’s architecture — they only have to provide relevant examples of cells representing a biologically meaningful group and the network will take care of the rest. However, the problem might lie in labelling of the data. Massive amounts of data are usually needed to properly train neural networks. Fortunately, machine learning provides tools to solve that problem.

First of all, there are methods to turn images into a small set of relevant numbers (i.e. the first stage of neural network-based processing) without providing any labels. They fall into the category of unsupervised or self-supervised methods and are a hot research topic in the machine learning community. Applying these techniques can address the first step in the convolutional neural network — turning images into numbers. It provides a form of pre-training that speeds up the learning process. Secondly, it turns out that although big amounts of data are generally required to train a network from scratch, some examples are more important than others. With the right selection, the number of examples required to train a network can be significantly reduced. Special algorithms are designed to choose appropriate examples. The scientist is engaged in an iterative process during which they observe examples with proposed labels.

At the beginning, the labels are random, so the scientist needs to correct them. During the next query, the algorithm tries to predict the classes of images that were not labelled before and chooses the ones it perceives as the most difficult. Another round of labelling follows. The scientist can end the process when they see that the query contains a satisfactory number of correct class assignments. The scientist had still only seen a small subset of data, so how can they be sure that all data was processed in a correct manner, in other words, whether the algorithm is generalised?

One way to check this is to visualise all data that was processed. You can use special techniques to represent each individual cell as a point on a 2-dimensional plot. Similar cells should be grouped together. Incorrectly labelled cells will form separate clusters that can be easily spotted and eliminated with a few mouse clicks.

This procedure doesn’t require the scientist to learn the intricacies of the deep neural networks that power such solution. In fact, they only have to transfer a part of their knowledge to the algorithm. The scientist is provided with an intuitive output that allows to easily spot outliers and unexpected patterns in the data. Thus, the neural network serves as the scientist’s unrelenting assistant able to learn from the scientist while only asking relevant questions.


We believe that the future lies in the human-machine cooperation. Machine learning algorithms won’t replace years of training required to design drugs, but can significantly help the scientists to digest massive amounts of data. At the same time, if combined with the right user interface, they save the scientists from months or years of learning about building, training and optimising deep neural networks. It’s only a matter of time when machine learning algorithms will be present in all stages of drug research, from early stage lab research to late stage clinical trials. If you want to learn about a real-life use of this technology, read the case study of a Machine Learning project we did for LifeArc.

Retail Report Resources
Marcel Falkiewicz Leader of Data Science
See all Marcel's posts

Related posts

You might be also interested in


Start your project with Objectivity

CTA Pattern - Contact - Middle

We use necessary cookies for the functionality of our website, as well as optional cookies for analytic, performance and/or marketing purposes. Collecting and reporting information via optional cookies helps us improve our website and reach out to you with information regarding our organisaton or offer. To read more or decline the use of some cookies please see our Cookie Settings.