Sprngy Platform Workload Management Documentation

Disclaimer

Sprngy Platform Documentation Guide

Release 2.0.0

This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.

This document is intended for:

SPRNGY Administrators
SPRNGY Developers
SPRNGY Architects

This document is a walkthrough to create and edit an Ingest Model through the Sprngy UI.

Pre-Requisites

Application is created in Admin UI using the Application Configuration screens and entities and relationships are defined.

Meta Models and Ingest Models are defined.

Data is imported (either through data upload UI or using import model UI)

Manage Workloads

Manage Workloads includes running workloads and Managing workloads. It keeps track of all the workloads that ran for which layer as well as the result of it.

Run Workload

Before running a workload, it is important to have appropriate data in the required data lake. By clicking on the run workload from the menu we get to the below page -

It requires details such as “Module Name”, “Entity Name” and “Rule From Layer” to run a workload for the selected data layer. On submit it will display a loader while running the workload in the background and when it is done will display a “SUCCESS” or “FAILURE” response on the screen.

The parameters that can be used as inputs for the “Rule From Layer” are as follows:

S.No.	Rule From Layer	Rule To Layer(Defaulted)
1.	SDL (Staging Data Lake)	FDL (Fast Data Lake)
2.	FDL (Fast Data Lake)	BDL (Business Data Lake)

The SDL → FDL workload executes pre-built data profiling pipelines as per the preferences provided in meta and ingest model. The FDL → BDL workload executes pre-built pipelines creating business datasets based on relationships and preferences defined in the meta and ingest model.

While running workloads, SDL → FDL workloads should be run first followed by FDL → BDL workloads.

Manage Workload

It is a table listing all the batches which have run previously.

The table displays Module Name, Entity Name, To Layer, From Layer, Batch_id, Workload Status, and Workload State for all the batches that run before. It also provides an option for a re-run and deleting batch.