Sprngy - Creating Demo Application using Connectors

1 Use Case Category: Social Media

This document details steps to set up new applications using Sprngy Admin UI, defining models, and running workloads.

Use Case Category: Social Media

Demo Application 1: Determine twitter user traffic in the USA

Objective - Build a chart that describes Twitter user traffic in the USA

Overview

Accessing twitter data using the Twitter developer account to gather information on tweets sent as per the geo-location. This search can be filtered by a hashtag, date and time, or geographical area. Here, we will be analyzing the number of tweets sent across the USA. We will be using “rtweet“ library to access the data and SprngyBI to create analytics over it.

Classifying the Application

Based on the data technology used and the business use, the application is classified as below:

Quality of Data	Data Lake Data Lake is a centralized repository to store large amount of raw data	Data Lakehouse / Data Warehouse Data Lakehouse combines concepts of Data Lake and Datawarehouse providing large storage capacity combined with data management features

Quality of Data	Data Lake Data Lake is a centralized repository to store large amount of raw data	Data Lakehouse / Data Warehouse
Curated
Correlated	✔
Normalized
Analyze	✔
Modelling

Step 1: Setting up the Application

Now that the business use and classification of the application are established, the application can be created using the UI. In AdminUI, set up the application by going to the Application Configuration page, select Create New, and filling out the file structure. Since we have just one layer data we will have just one entity in it.

Step 2: Create Analytic Model

Since we are accessing data using the Tweeter API we can use Analytic Model directly to load, correlate and analyze the data.

Analytic model can be specified using the AdminUI. A Sample Analytic Model is as follows:

Module Name	Processor Name	Pipeline Sequence	Variable Name	Assign Type	Parameter

Module Name	Processor Name	Pipeline Sequence	Variable Name	Assign Type	Parameter
Tweeter	tweeter_data	1	token	CREATE_VAR	rtweet::create_token(app = "BA_Demo123", consumer_key = 'pS10oUlc2s4Y70jX2XFXUGniL', consumer_secret = 'Ru2IVaCNyZsUM7UVHXZ0QFvma7BRrjL3J5xXvmeaqneOleT1q6', access_token = '1546529453131587584-No2NRwnvjWmXpsTDS71yjvTi23r6NQ', access_secret = '4lP01ELVNvqaF5kPMyJM80KwkHoFIxID0Az7jKEVZKuXw')
Tweeter	tweeter_data	2	data	CREATE_VAR	rtweet::search_tweets("lang:en", geocode = lookup_coords("usa"), n = 10000, token = token)
Tweeter	tweeter_data	3	data_lat_lng	CREATE_VAR	rtweet::lat_lng(data)
Tweeter	tweeter_data	4	res	CREATE_VAR	data_lat_lng %>% dataops.overwritedata('/BigAnalytixsPlatform/Tweeter/tweeter_data/BAL','data')

This would fetch the tweeter data, add latitude and longitude to it and save it to the given location.

Step 3: Create Data Table in Hive

For now if you run an analytic model please contact to support@sprngy.com to get the generated data access in SprngyBI.

Step 4: Importing Database and Dataset into SprngyBI

After adding the database and dataset (as shown in above screenshots), we can create charts for that dataset. To create charts go to the Chart page from the top menu, then pick the dataset that we just add and select the chart type. Here, we are selecting “deck.gl Scatterplot“ this requires latitude and longitude columns.