...
You can consolidate multiple Meta files into one using consolidate functionality. This is assuming that the schema of all files in the meta-model folder is the same.
Can I consolidate Meta files?
You can consolidate multiple Meta files into one using consolidate functionality. This is assuming that the schema of all files in the meta-model folder is the same.
What if there is Parent-child relationship in your data module?
...
You need to update the field (through which you are planning to make a join with the parent entity) of your child entity in meta model by updating the “Entity Attribute Compare Key Role“ to “Yes”, “Entity Attribute Parent Lookup Key Role“ to “Yes” and in “Entity Attribute Parent Lookup Location“ you need to give the FDL/Stage location of your parent entity. Please refer above screenshot, where “movies” is the parent entity and “roles” is the child entity.
How can I create custom transform rules in the Meta Model?
Transform rules are SQL queries you can write in order to filter data by a variable, change certain values in a column, and apply the other functions that are in the Spark SQL library. We can write the transform rules in the meta model by clicking the edit icon next to the column name and going to the Rules page.
...
Here are a couple things to keep in mind when writing your transform rules:
Do not put ‘Select’ at the beginning of the query, as it is appended automatically to the front when the backend processor runs
The SQL query should end in “as <column_name>”
The transform rules that you define will be executed in the SDL-FDL workload when you define the TRANSFORM_RECORDS_PROCESSOR in the Ingest Model