Automation of Sample Analysis for Scientific Research Biolaboratory

Project Description
The client is a biotechnological laboratory based at a major university, with a focus on tissue engineering and biotechnologies. The client has a vast amount of raw data obtained from spectrophotometers, confocal microscopes, EEG/ECG/spirometry of laboratory animals, and other unprocessed results. As part of scientific grants, the laboratory collaborates with several universities from the USA and Europe.

Client's Main Requests: Development and implementation of a fully automated process for analyzing stained tissue samples. A typical task involves a researcher in one country uploading 4,000 microphotographs into the system, and a researcher in another country receiving a finished document with detailed statistical analysis and data on the studied sample. Implementation of JupiterHub for performing routine tasks. Ensure primary migration from Matlab and GNU Octave to a stack consisting of NumPy, SciPy, matplotlib/seaborn.

Key Metrics
  • 27% reduction in labor efforts by highly qualified research personnel
  • 6 teams in different cities and countries
  • 20+ different data sources, including custom-made, single-instance devices
  • 10,000+ samples for each series of experiments
Key Challenges and Results
To address the issue, we involved an expert from our team who had previously worked in a scientific laboratory with similar tasks. The main challenge faced by the client was the correct segmentation of areas that were simultaneously stained with multiple dyes, followed by morphometric analysis and the labor-intensive manual data processing. We suggested the use of a color deconvolution method based on the paper "Ruifrok AC, Johnston DA. Quantification of histochemical staining by color deconvolution" implemented in Python code.
Results of the implemented utility
The proposed MVP produced results that fully satisfied the client in terms of segmentation quality, specificity, and sensitivity. We optimized the final solution for GPU use, significantly enhancing performance. Based on this solution, we established a workflow with automatic processing of samples from the uploaded pool and exporting the obtained pandas dataframes as processed tabular data. Additionally, we made several improvements for detecting objects with specific parameters in the images to enrich statistical data.
The second major challenge was the initial adaptation of the client's toolset from GNU Octave to the Python stack consisting of NumPy, SciPy for collaborative work in JupiterHub. The client decided to change the stack since a limited number of employees were proficient in Matlab, complicating the joint use of the developed toolkit. We deployed a JupiterHub installation on a Kubernetes cluster with quota allocation through PersistentVolumeClaims. To facilitate migration, we compiled and analyzed a series of typical data processing tasks previously solved with Octave, for example, typical tasks for analyzing raw data with cyclo-periodic tests of materials on the Instron 8801 device, spectral analysis of electromyogram recordings. We developed documentation, standard Docker images with the proper environment, and helped rewrite typical tasks using NumPy, SciPy libraries. For example, for Fast Fourier Transform of unprocessed data requiring spectral analysis.
As a result of our work:
  1. Reduced labor costs of laboratory personnel on routine project tasks by 27%.
  2. Fully automated pipeline from microphotograph upload to making analyzed sample data available for the teams, including quasi-proteomic analysis.
  3. Provided migration of the main toolkit to more common standard Python libraries, enabling more scientists to participate in their development.
  4. Prepared a platform for launching JupiterNotebook with a ready-to-use environment, understandable to researchers needing data analysis but not developers.
Related services
Comprehensive CI/CD Services | CI Consulting & Automation | WiseOps
Optimize your software delivery with WiseOps' CI/CD services. From CI consulting to CD construction consulting, we offer continuous integration solutions and CI/CD as a service to streamline your development process. Contact us for professional CI/CD implementation and consulting.
Containerization and Orchestration Services - WiseOps Team
Discover scalable and fault-tolerant containerization and orchestration services with WiseOps. Optimize resource usage and accelerate deployment with our expert solutions in container orchestration and service orchestration in cloud computing.