| Table of Contents | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Disclaimer
Sprngy Platform Documentation Guide
Release 2.0.0
Copyright © Sprngy Corporation. All rights reserved.
This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.
...
This document is a walkthrough to create and edit an Ingest Model through the Sprngy UI.
Pre
...
The list of tools required to run/develop in the UI are as follows:
Visual Studio Code (Developers only)
Python 3.8.10
Flask 1.1.4
Node 14.17.6
npm 7.24.1
R 3.6.3
Sprngy Libraries
Requisites
Application is created in Admin UI using the Application Configuration screens and entities and relationships are defined.
Meta Models and Ingest Models are defined.
Data is imported (either through data upload UI or using import model UI)
Manage Workloads
Manage Workloads includes running workloads and Managing workloads. It keeps track of all the workloads that ran for which layer as well as the result of it.
...
S.No. | Rule From Layer | Rule To Layer(Defaulted) |
|---|---|---|
1. | SDL (Staging Data Lake) | FDL (Fast Data Lake) |
2. | FDL (Fast Data Lake) | BDL |
3. | BDL | BAL |
...
(Business Data Lake) |
The SDL → FDL workload executes pre-built data profiling pipelines as per the preferences provided in meta and ingest model. The FDL → BDL workload executes pre-built pipelines creating business datasets based on relationships and preferences defined in the meta and ingest model.
While running workloads, SDL → FDL workloads should be run first followed by FDL → BDL workloads.
Manage Workload
It is a grid having table listing all the batches which have run previously.
...
The Grid table displays Module Name, Entity Name, To Layer, From Layer, Batch_id, Workload Status, and Workload State for all the batches that run before. It also provides an option for a re-run and deleting batch.
Copyright © Springy Corporation. All rights reserved. Not to be reproduced or distributed without express written consent.