Skip to main content

Overview

This document provides a high-level overview of the architecture of our SAAS platform for Public Data Due Diligence and Third Party Risk Management (TPRM).

System Architecture​

Our platform comprises three main systems that work together to achieve our set objectives:

  • Data Aggregation and Integration System (DAIS) - This system is responsible for public data scraping, building and maintaining data warehouses, and integrating with third-party data providers.

  • Report & Analysis Execution System (RAES) - This system automates the due diligence report preparation process. It enables analysts to configure reports, monitor report execution, and perform manual data entry, among other functions.

  • Client App - This is the application (frontend and its backend) used by client users to place requests, explore and consume reports, manage third party lists, and monitor risk signals.

Technology Stack​

Our platform employs a robust and flexible technology stack:

  • Backend: Python
  • Frontend: React.js (with material-ui library)
  • Microservices: Nameko
  • Client App backend, APIs, and Database: Django, Django Rest Framework, PostgreSQL
  • Public data scraping: Scrapy, Selenium, Python requests
  • Containerization and dependency management: Docker, supervisord, poetry
  • Data Warehouse: PostgreSQL with Sqlalchemy
  • Caching: Redis
  • Cloud Deployment: AWS services (s3, ec2, rds, route53, alb, aws cdk)
  • CI/CD: GitLab
  • ML model training: Pytorch
  • Testing: pytest (backend), storybook (frontend)
  • Service Discovery and Communication: RabbitMQ

Interactions and Communications between the Systems​

The three systems interact and communicate with each other to function as a cohesive platform:

  • DAIS ↔ RAES: The DAIS sources public data, transforms it, and loads it into a data warehouse. The RAES then accesses this data via data services to conduct necessary analysis as part of the due diligence report preparation.

  • RAES ↔ Client App: Analysts use the RAES to create and manage reports based on the client users' requests made via the Client App. The reports are then delivered back to the Client App for client users to access and explore.

  • DAIS ↔ Client App: In some instances, client users may need to directly access data assets in our data warehouses. This can be done through API integration or via micro apps within the Client App.

All interactions between the systems are facilitated either using RabbitMQ, a robust messaging system that supports service discovery and communication or through or Standard Rest APIs. In addition, we use Docker for containerization, enabling us to run our applications in a loose-coupled, distributed environment for scalability and reliability.

Our SAAS platform's architecture is designed to be robust, efficient, and scalable, ensuring seamless TPRM and Public Data Due Diligence processes. It combines the power of automated workflows, data aggregation, and user-friendly interfaces to deliver actionable insights for efficient decision-making.