Disclaimer
Sprngy Platform Azure Engineering Guide
Release 1.0.0.1
Copyright © Sprngy. All rights reserved.
This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.
This document is intended for:
SPRINGYCORE Administrators
SPRINGYCORE Developers
SPRINGYCORE Architects
This document is a walkthrough to create and edit an Ingest Model through the Sprngy UI.
Pre-requisites:
The list of tools required to run/develop in the UI are as follows:
Visual Studio Code (Developers only)
Python 3.8.10
Flask 1.1.4
Node 14.17.6
npm 7.24.1
R 3.6.3
Sprngy Libraries
Ingest Model:
An ingest model is used to define rules for processing the entity data. Below are the processes to Create and Edit a Meta Model from the UI.
Creating an Ingest Model:
Home Screen:
The image below is the Home screen of the UI. From the side menu open ‘Meta Data Configuration' option, then select 'Ingest Model’ option.
Create Ingest:
Step 1 -
After clicking on the Ingest Model menu option below screen will be displayed.
To create a new Ingest Model, select the required Module name and Entity name from the drop down and click 'Submit'.
This checks the existence of a ingest model. If there is a previously created or even initiated ingest model then it will bring back that data, if creating ingest model for the very first time then provides blank fields for input.
Step 2 -
After submission, the below screen will be rendered. Here you can provide information regarding the ingest model for given entity.
This two step form takes information such as entity description and business owner, data owner and IT owner information working on this entity. Once done click on Submit.
Field Name | Description | Data Type | is Required? | Validation
|
|---|---|---|---|---|
Module Name | Choose from the list of modules | String | Yes | One of the seven on the website |
Entity Name | Choose from the list of entities | String | Yes | Only appears after the module name is selected |
Entity Description | The description of the respective entity | String | Yes |
|
Field Name | Description | Data Type | is Required? | Validation |
|---|---|---|---|---|
Entity Business Owner Name | First name + Last name of Entity Business Owner | String | Yes |
|
Entity Business Owner Email | The Business Owner email Id | String | Yes | should have @ |
Entity Data Owner Name | The Data Owner Name of Entity | String | Yes |
|
Entity Data Owner Email | The Data Owner Email of Entity | String | Yes | should have @ |
Entity It Owner Name | The IT owner name of an entity | String | Yes |
|
Entity It Owner Email | The IT owner email of the entity | String | Yes | should have @ |
Vendor Support Name | The vendor support name of the entity.Default Non-editable to support@biganalytixs.com | String | Auto-Fill |
|
Vendor Support Email | The vendor support name of the entity.Default Non-editable to support@biganalytixs.com | String | Auto-Fill |
|
After these details have been filled out, you can hit on “Back” to go to “Step 1” or if you are done filling out your PreEntity Details you can hit on “Submit”.
Step 3 -
On submit, you will get a form with actions to be taken while processing data. It consist of a list of questions such as ways of reading data, remove nulls from dataset, impute the nulls with default value and so on.
After filling the details, click on 'Submit'. This would save the ingest model details.
Editing an Ingest
Editing a Ingest model has similar steps to creating a Ingest Model. From the side menu open ‘Meta Data Configuration' option, then select 'Ingest Model’ option.
Step 1 -
After clicking on the Ingest Model menu option below screen will be displayed.
Here select the module and entity of which ingest model you need to update and click on submit button.
This will load the information you previously stored for that module and entity. First of all there is the entity owner information that you can update. After that there will the form describing the rules to be applied while processing data.
Step 2 -
On Submitting the owner information, below screen will be loaded.
Here, the rules can be updated as required and the click on 'Submit'. This will save the changes made to the ingest model.
Bulk Upload Ingest Model
There is also one other way to create a ingest model, that is by uploading a CSV file. For this, you are required to have a ingest model .csv file with all the required fields. To directly upload a csv -
Home Screen:
The image below is the Home screen of the UI.
From the side menu open ‘Meta Data Configuration' option, then select 'Ingest Model’ option.
Step 1 -
After clicking on the Ingest Model menu option below screen will be displayed.
Select the module and entity for which you want to upload a ingest model. In the file upload option browse or drag and drop the ingest model csv file and click Submit.