| Table of Contents | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Disclaimer
Sprngy Platform Azure Engineering Documentation Guide
Release 12.0.0.1
Copyright © Sprngy Corporation. All rights reserved.
This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.
This document is intended for:
SPRNGYCORE SPRNGY Administrators
SPRNGYCORE SPRNGY Developers
SPRNGYCORE SPRNGY Architects
This document is a walkthrough to create and edit a Meta Model through the Sprngy UI.
Pre
...
The list of tools required to run/develop in the UI are as follows:
Visual Studio Code (Developers only)
Python 3.8.10
Flask 1.1.4
Node 14.17.6
npm 7.24.1
R 3.6.3
Big Analytixs Libraries
...
Requisites
Application is created in Admin UI using the Application Configuration screens and entities and relationships are defined.
Define Meta Model
A meta model is a format that allows to captured capture detailed information about attributes/columns of data sets that are to be processed. Below are the processes steps to Create/Edit a Meta Model from the UI.
...
The unique field while creating an attribute is “entity_attribute_model_name”. You will get an error if you try to create an attribute with the same name.
When the meta model is created, a field called “as_of_date” will be system generated and defaulted. You will not be able to edit or delete that field.
Home Screen
...
The image below is On the Home screen of the UI.
Click on the three lines (pulls up the menu) and click on the “Create/Edit Meta Model”
...
Screen 0:
...
This admin UI, click ‘Meta Data Configuration' option on side menu and then click 'Meta Model’.
...
Step 1
After clicking on the Meta Model menu option below screen will be displayed.
...
To create a new Meta Model, select the required Module name and Entity name from the drop down and click 'Submit'.
Alternatively, using the Upload box, meta model (in csv) format can be uploaded for the selected entity.
On clicking ‘Submit’, Admin UI checks the existence of a metamodel. There are a few checks taking place:First, the local JSON files are checked for. If it does not exist(implies that you have not left creating/editing the metamodel halfway), it will then check the HDFS for that meta model. If it exists in the HDFS, then that existing metamodel is pulled. If not, then the metamodel will be created from scratch.If there is a previously created or even initiated meta model then it will bring back that data, if creating meta model for the very first time then provides blank fields for input.
Step 2
After submission, the below screen will be rendered. Here you can provide information regarding the meta model for given entity.
...
This two step form takes information such as entity description and business owner, data owner and IT owner information working on this entity. Once done click on Submit.
Step 3
On the next screen there will be fields to add attribute name (i.e. Column Name) and it’s datatype. This screen is visible only when creating the meta model for first time. Add all the column names with it’s datatypes (string,numeric or integer) and click Submit. On submit, a meta model will be created with default values for the given values.
...
It will by default add a column ‘as_of_date'. You can also skip this page which would only add the 'as_of_date' column.
...
Step 4
The next page rendered will be of a grid with created column names. To customise customize or change any configurations click on edit icon of that column name.
Attributes Grid Screen:
Clicking on the “Submit” button on the Form1 page, will save your preEntity details locally and route you to the Attributes/Grid Screen. In this screen you have the list of attributes that exist in the meta model.
...
There are multiple functionalities that are available on this screen:1.
Add New Attribute
...
Form2(AttributeForm):
You can only fill and save one attribute at a time. The following is a 5 step page to incorporate all the details required to create one attribute:
Step 1:
...
It is provides similar option as before to add attribute name and data type.
...
Edit Attribute
The Edit icon gives you the ability to edit that particular attribute and save it at every step of the way.
...
Here are the fields that can be configured on this screen:
Field Name | Description | Data Type | Is Required? | Validation |
|---|---|---|---|---|
Entity Attribute Model Name | The name of the attribute | String | Yes | No spaces |
Entity Attribute Model data format | The format of the attribute - character, numeric, or integer | String | Yes | one of the three in column B |
Entity Attribute Model Data Length | The length of the field | Numeric | Yes | only positive numbers |
Entity Attribute Model Data Precision | if numeric then precision | Numeric | Yes | only positive numbers |
Entity Attribute Unique Ind | Yes/No Indicator | (Yes or No flag) | Yes | YES/NO |
Entity Attribute In Use Ind | Yes/No Indicator | (Yes or No flag) | Yes | YES/NO |
Entity Attribute Description | The metadata definition of the attribute | String | Yes |
|
Entity Attribute Sample Value | The sample value of the attribute | String | Yes | Length of field should be same as input of “Entity Attribute Model Data Length” |
Entity Attribute Default Value | The default value to assign if not imputing to treat nulls | String | Yes | Length of field should be same as input of “Entity Attribute Model Data Length” |
Step 2:
...
Define Transform, Quality and Validation rules
...
Here are the fields that can be configured on this screen:
Field Name | Description | Data Type | Is Required? | Validation |
|---|---|---|---|---|
Entity Attribute Transform Rule Inbound | The transform rule for the attribute | String | Yes |
|
Entity Attribute Transform Rule Outbound | The transform rule for the attribute | String | Yes | by default |
Entity Attribute Validation Rule | the validation rule for the attribute | String | Yes | by default NONE |
Entity Attribute Validation Rule Description | the validation rule for the attribute | String | Yes | by default NONE |
Entity Attribute Validation Rule Iterate by Key Role | Yes/No Indicator | String | Yes | YES/NO |
Entity Attribute Quality Rule | a~b | String | Yes | Has to have “~” in the rule or “NONE” if there is no quality rule |
Entity Attribute Quality Rule Description | a~b | String | Yes |
|
Entity Attribute Quality Rule Iterate By Key Role | other than a or b | String | Yes |
|
Step 3:
...
Define Relationships
...
Here are the fields that can be configured on this screen:
Field Name | Description | Data Type | Is Required? | Validation |
|---|---|---|---|---|
Entity Attribute Impute Method | if the attribute is null, pick out an impute method. For character columns, it's the default value. For numeric nd integers, mean or median | String | Yes | one of the three in column B |
Entity Attribute Compare Key Role | Yes/No Indicator | String | Yes | YES/NO |
Entity Attribute Filter Key | the construct to filter records | String | Yes | default is NO |
Entity Attribute Nested Key Role | Whether the given attribute should be nested used in nesting operatio or not | String | Yes | default is YES |
Entity Attribute Parent Lookup Key Role | (Yes or No flag) | String | Yes | YES/NO |
Entity Attribute Parent Lookup Location | The hdfs location of the parent entity for performing lookups | String | Yes | default is NONE |
Entity Attribute Calendar join key Role | (Yes or No flag) | String | Yes | YES/NO |
Step 4:
...
Define Auditing Preferences
...
Here are the fields that can be configured on this screen:
Field Name | Description | Data Type | Is Required? | Validation |
|---|---|---|---|---|
Entity Attribute Iqm Codes Match | (Yes or No flag) | String | Yes | YES/NO |
Entity Attribute Iqm Metric Match | (Yes or No flag) | String | Yes | YES/NO |
Entity Attribute Iqm Iterate By Key Role | (Yes or No flag) | String | Yes | YES/NO |
Entity Attribute Eda Dimension | (Yes or No flag) | String | Yes | YES/NO |
Entity Attribute Eda Metric | (Yes or No flag) | String | Yes | YES/NO |
Entity Attribute Eda Iterate By Key Role | (Yes or No flag) | String | Yes | YES/NO |
Step 5:
...
Define Attribute Presentation on Query and Report Layer
...
Here are the fields that can be configured on this screen:
Field Name | Description | Data Type | Is Required? | Validation |
|---|---|---|---|---|
Entity BI Object Repository | defaults to the module name | String | Yes | NO Space |
Entity BI Object Name | defaults to, The name of the attribute | String | Yes | NO Space |
Entity Attribute BI Name | defaults to , The format of the attribute - character, numeric, or integer- can be editable | String | Yes | NO Space |
Entity Attribute BI Mask Ind | (Yes or No flag) | String | Yes | YES/NO |
Entity Attribute BI Mask | Whether or not to Mask the attribute | String | Yes |
|
Entity Attribute BI Data Format | The format of the attribute - character, numeric, or integer | String | Yes |
|
Entity Attribute BI Data Length | The length of the field | String | Yes | Number |
Entity Attribute BI Data Precision | if numeric then precision | String | Yes | Number |
...
After you are done filling out the details for a single attribute, click the “SAVE ATTRIBUTE” button to save the file locally and route you back to the Attributes Grid Page.2. The View Icon -
View Attribute
This icon navigates you to another screen which displays the details of the particular attribute.
...
...
Delete Icon
The Delete Icon gives you the ability to delete that particular attribute. A pop up will appear reconfirming your decision to delete the attribute.
After making all changes to the meta model, you can click on the “Final Submit” button in order to update the existing meta model in the HDFS. It is IMPORTANT to remember that you have to click on the “FINAL SUBMIT” button in order for your changes to be permanently saved.
Edit Meta Model
Editing a Meta model has similar steps to creating a Meta Model.
Home Screen
...
The image below is the Home screen of the UI.
Click on the three lines (pulls up the menu) and click on the “Create/Edit Meta Model”
...
Screen 0:
...
This checks the existence of a metamodel. There are a few checks taking place:
First, the local JSON files are checked for. If it does not exist(implies that you have not left creating/editing the metamodel halfway), it will then check the HDFS for that meta model. If it exists in the HDFS, then that existing metamodel is pulled. If not, then the metamodel will be created from scratch.
On Submit, you will be routed to the MainForm.
Form1(MainForm):
The Mainform requires you to fill out the preEntity details and is a page with two steps. The following is the “Step 1”:
...
Field Name
...
Description
...
Data Type
...
is Required?
...
Validation
...
Module Name
...
Choose from the list of modules
...
String
...
Yes
...
One of the seven on the website
...
Entity Name
...
Choose from the list of entities
...
String
...
Yes
...
Only appears after the module name is selected
...
Entity Description
...
The description of the respective entity
...
String
...
Yes
...
...
Entity Type
...
Fact
...
String
...
Auto Fill
...
Once you click on “Next”, you will be sent to “Step 2” to fill out the other preEntity details. The following image is “Step 2”:
...
Field Name
...
Description
...
Data Type
...
is Required?
...
Validation
...
Field Name
...
Description
...
Data Type
...
is Required?
...
Validation
...
Entity Business Owner Name
...
First name + Last name of Entity Business Owner
...
String
...
Yes
...
...
Entity Business Owner Email
...
The Business Owner email Id
...
String
...
Yes
...
should have @
...
Entity Data Owner Name
...
The Data Owner Name of Entity
...
String
...
Yes
...
...
Entity Data Owner Email
...
The Data Owner Email of Entity
...
String
...
Yes
...
should have @
...
Entity It Owner Name
...
The IT owner name of an entity
...
String
...
Yes
...
...
Entity It Owner Email
...
The IT owner email of the entity
...
String
...
Yes
...
should have @
...
Vendor Support Name
...
The vendor support name of the entity.Default Non-editable to support@biganalytixs.com
...
String
...
Auto-Fill
...
...
Vendor Support Email
...
The vendor support name of the entity.Default Non-editable to support@biganalytixs.com
...
String
...
Auto-Fill
...
After these details have been filled out, you can hit on “Back” to go to “Step 1” or if you are done filling out your PreEntity Details you can hit on “Submit”.
Attributes Grid Screen:
Clicking on the “Submit” button on the Form1 page, will save your preEntity details locally and route you to the Attributes/Grid Screen. In this screen you have the list of attributes that exist in the meta model.
...
There are multiple functionalities that are available on this screen:
Add New Attribute
- It navigates you to the Form2 as explained below:
Form2(AttributeForm):
You can only fill and save one attribute at a time. The following is a 5 step page to incorporate all the details required to create one attribute:
Step 1:
...
Field Name
...
Description
...
Data Type
...
Is Required?
...
Validation
...
Entity Attribute Model Name
...
The name of the attribute
...
String
...
Yes
...
No spaces
...
Entity Attribute Model data format
...
The format of the attribute - character, numeric, or integer
...
String
...
Yes
...
one of the three in column B
...
Entity Attribute Model Data Length
...
The length of the field
...
Numeric
...
Yes
...
only positive numbers
...
Entity Attribute Model Data Precision
...
if numeric then precision
...
Numeric
...
Yes
...
only positive numbers
...
Entity Attribute Unique Ind
...
Yes/No Indicator
...
(Yes or No flag)
...
Yes
...
YES/NO
...
Entity Attribute In Use Ind
...
Yes/No Indicator
...
(Yes or No flag)
...
Yes
...
YES/NO
...
Entity Attribute Description
...
The metadata definition of the attribute
...
String
...
Yes
...
...
Entity Attribute Sample Value
...
The sample value of the attribute
...
String
...
Yes
...
Length of field should be same as input of “Entity Attribute Model Data Length”
...
Entity Attribute Default Value
...
The default value to assign if not imputing to treat nulls
...
String
...
Yes
...
Length of field should be same as input of “Entity Attribute Model Data Length”
Step 2:
...
Field Name
...
Description
...
Data Type
...
Is Required?
...
Validation
...
Entity Attribute Transform Rule Inbound
...
The transform rule for the attribute
...
String
...
Yes
...
...
Entity Attribute Transform Rule Outbound
...
The transform rule for the attribute
...
String
...
Yes
...
by default
...
Entity Attribute Validation Rule
...
the validation rule for the attribute
...
String
...
Yes
...
by default NONE
...
Entity Attribute Validation Rule Description
...
the validation rule for the attribute
...
String
...
Yes
...
by default NONE
...
Entity Attribute Validation Rule Iterate by Key Role
...
Yes/No Indicator
...
String
...
Yes
...
YES/NO
...
Entity Attribute Quality Rule
...
a~b
...
String
...
Yes
...
Has to have “~” in the rule or “NONE” if there is no quality rule
...
Entity Attribute Quality Rule Description
...
a~b
...
String
...
Yes
...
...
Entity Attribute Quality Rule Iterate By Key Role
...
other than a or b
...
String
...
Yes
...
Step 3:
...
...
Field Name
...
Description
...
Data Type
...
Is Required?
...
Validation
...
Entity Attribute Impute Method
...
if the attribute is null, pick out an impute method. For character columns, it's the default value. For numeric nd integers, mean or median
...
String
...
Yes
...
one of the three in column B
...
Entity Attribute Compare Key Role
...
Yes/No Indicator
...
String
...
Yes
...
YES/NO
...
Entity Attribute Filter Key
...
the construct to filter records
...
String
...
Yes
...
default is NO
...
Entity Attribute Nested Key Role
...
Whether the given attribute should be nested or not
...
String
...
Yes
...
default is YES
...
Entity Attribute Parent Lookup Key Role
...
(Yes or No flag)
...
String
...
Yes
...
YES/NO
...
Entity Attribute Parent Lookup Location
...
The hdfs location of the parent entity for performing lookups
...
String
...
Yes
...
default is NONE
...
Entity Attribute Calendar join key Role
...
(Yes or No flag)
...
String
...
Yes
...
YES/NO
Step 4:
...
Field Name
...
Description
...
Data Type
...
Is Required?
...
Validation
...
Entity Attribute Iqm Codes Match
...
(Yes or No flag)
...
String
...
Yes
...
YES/NO
...
Entity Attribute Iqm Metric Match
...
(Yes or No flag)
...
String
...
Yes
...
YES/NO
...
Entity Attribute Iqm Iterate By Key Role
...
(Yes or No flag)
...
String
...
Yes
...
YES/NO
...
Entity Attribute Eda Dimension
...
(Yes or No flag)
...
String
...
Yes
...
YES/NO
...
Entity Attribute Eda Metric
...
(Yes or No flag)
...
String
...
Yes
...
YES/NO
...
Entity Attribute Eda Iterate By Key Role
...
(Yes or No flag)
...
String
...
Yes
...
YES/NO
Step 5:
...
Field Name
...
Description
...
Data Type
...
Is Required?
...
Validation
...
Entity BI Object Repository
...
defaults to the module name
...
String
...
Yes
...
NO Space
...
Entity BI Object Name
...
defaults to, The name of the attribute
...
String
...
Yes
...
NO Space
...
Entity Attribute BI Name
...
defaults to , The format of the attribute - character, numeric, or integer- can be editable
...
String
...
Yes
...
NO Space
...
Entity Attribute BI Mask Ind
...
(Yes or No flag)
...
String
...
Yes
...
YES/NO
...
Entity Attribute BI Mask
...
Whether or not to Mask the attribute
...
String
...
Yes
...
...
Entity Attribute BI Data Format
...
The format of the attribute - character, numeric, or integer
...
String
...
Yes
...
...
Entity Attribute BI Data Length
...
The length of the field
...
String
...
Yes
...
Number
...
Entity Attribute BI Data Precision
...
if numeric then precision
...
String
...
Yes
...
Number
After you are done filling out the details for a single attribute, click the “SAVE ATTRIBUTE” button to save the file locally and route you back to the Attributes Grid Page.
The View Icon
- This icon navigates you to another screen which displays the details of the particular attribute.
...
Edit Icon -
The Edit icon gives you the ability to edit that particular attribute and save it at every step of the way.
Delete Icon -
The Delete Icon gives you the ability to delete that particular attribute. A pop up will appear reconfirming your decision to delete the attribute.
After making all changes to the meta model, you can click on the “Final Submit” button in order to update the existing meta model in the HDFS. It is IMPORTANT to remember that you have to click on the “FINAL SUBMIT” button in order for your changes to be permanently savedFrom the side menu open ‘Meta Data Configuration' option, then select 'Meta Model’ option.
...
Step 1
After clicking on the Meta Model menu option below screen will be displayed.
...
Select the module and entity you want to edit from the drop down and click Submit.
This will load the information you previously stored for that module and entity. First of all there is the entity owner information that you can update. After that there will a list of all the attributes.
You can Edit, View, Delete and Add attributes similar to the Create Meta Model flow as discussed above.
Once satisfied with the Meta Model changes click on 'Final Submit' to save these changes.
Copyright © Springy Corporation. All rights reserved. Not to be reproduced or distributed without express written consent.