Skip to main content

Interactions and Communications between the Systems

This section explores the communication and interaction pathways between the three primary systems of our platform: the Data Aggregation and Integration System (DAIS), the Report & Analysis Execution System (RAES), and the Client App.

DAIS ↔ RAES Interaction​

The DAIS is the primary source of data for the RAES. Once the DAIS scrapes public data and populates the data warehouse, the RAES accesses this data for its analysis processes. This interaction is critical for the automated report generation and due diligence workflows.

In more detail, the interaction process includes:

  • Data Collection and Warehousing: The DAIS collects public data using Scrapy, Selenium, and Python requests. The scraped data undergoes transformation via ETL pipelines and is stored in a PostgreSQL database managed with Sqlalchemy.

  • Data Access: The RAES retrieves the data through data services. These data services ensure secure and efficient access to the data, maintaining data integrity and preventing unauthorized access.

RAES ↔ Client App Interaction​

The RAES and the Client App interact continuously for the creation, management, and delivery of reports based on client users' requests. The interaction process can be broken down into the following steps:

  • Report Request: Client users place requests for reports or continuous monitoring through the Client App. This request data is transferred to the RAES.

  • Report Configuration and Generation: The RAES uses the request data and additional data from the DAIS to configure and generate the required report. Analyst users assist in this process through the analyst interface.

  • Report Delivery: The completed reports are then sent back to the Client App where client users can access, explore, and download them.

DAIS ↔ Client App Interaction​

The DAIS and Client App also communicate directly, particularly when client users require direct access to data assets in the data warehouse. The interaction process includes:

  • Data Access Request: Client users, via the Client App, may require access to specific data assets. These requests are sent to the DAIS.

  • Data Retrieval: The DAIS fetches the requested data from the data warehouse.

  • Data Delivery: The requested data is delivered back to the Client App, either via API integration for seamless integration with user workflows or via micro apps within the Client App.

Facilitating Interactions​

RabbitMQ and standard Rest APIs facilitates all interactions between the systems. This robust message broker supports asynchronous messaging, enabling independent and loose coupling of the services. Docker is employed for containerization, promoting isolated environments for each service and ensuring their smooth operation without interference.

These interactions and communications between the systems are central to the functioning of our platform. They ensure efficient data flow and seamless cooperation between different components, leading to robust and timely TPRM and Public Data Due Diligence processes.

Top Level Components and their Interactions​

architecture