| Table of Contents | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Disclaimer
Sprngy Platform Azure Engineering Documentation Guide
Release 12.0.0.1
Copyright © Sprngy Corporation. All rights reserved.
This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.
This document is intended for:
BAPCORE SPRNGY Administrators
BAPCORE SPRNGY Developers
BAPCORE SPRNGY Architects
This document is a walkthrough to run, re-run and delete workload through Sprngy UI
Pre-requisites:
The list of tools required to run/develop in the UI are as follows:
Visual Studio Code (Developers only)
Python 3.8.10
Flask 1.1.4
Node 14.17.6
npm 7.24.1
R 3.6.3
Big Analytixs Libraries
Batch Management
Batch management create and edit an Ingest Model through the Sprngy UI.
Pre Requisites
Application is created in Admin UI using the Application Configuration screens and entities and relationships are defined.
Meta Models and Ingest Models are defined.
Data is imported (either through data upload UI or using import model UI)
Manage Workloads
Manage Workloads includes running workloads and Managing workloads. It keeps track of all the workloads that ran for which layer as well as the result of it.
...
Run Workload
Before running a workload, it is important to have appropriate data in the required data lake. By clicking on the run workload from the menu we get to the below page -
...
S.No. | Rule From Layer | Rule To Layer(Defaulted) |
|---|---|---|
1. | SDL (Staging Data Lake) | FDL (Fast Data Lake) |
2. | FDL (Fast Data Lake) | BDL |
3. | BDL | BAL |
...
(Business Data Lake) |
The SDL → FDL workload executes pre-built data profiling pipelines as per the preferences provided in meta and ingest model. The FDL → BDL workload executes pre-built pipelines creating business datasets based on relationships and preferences defined in the meta and ingest model.
While running workloads, SDL → FDL workloads should be run first followed by FDL → BDL workloads.
Manage Workload
It is a grid having table listing all the batches which have run previously.
...
The Grid table displays Module Name, Entity Name, To Layer, From Layer, Batch_id, Workload Status, and Workload State for all the batches that run before. It also provides an option for a re-run and deleting batch.
Copyright © Springy Corporation. All rights reserved. Not to be reproduced or distributed without express written consent.