What is your DANL 101 session schedule?

MWF 9:30am-10:20am TTh 3:30pm-4:45pm

What is your first name?

What is your last name?

Please enter your Geneseo email address without @geneseo.edu. For example, if your full Geneseo email address is abcd234@geneseo.edu, you should enter only abcd234. If you do not have a Geneseo email, please provide your full email address (e.g., abcd@gmail.com).

What is your GitHub username?

How many times have you submitted this assignment so far?

1. A data lake stores data in its raw, unaltered form and uses a schema-on-read approach. (True/False)

True False

2. Apache Hadoop can natively process real-time streaming data without any extensions. (True/False)

True False

3. The Extract, Transform, Load (ETL) process involves extracting data from multiple sources, transforming it into a suitable format, and loading it into a data warehouse. (True/False)

True False

4. Data marts are subsets of data warehouses that are focused on specific business areas or departments. (True/False)

True False

5. 'Structured data' refers to data that is not organized in a predefined manner and comes from sources like social media, emails, and videos. (True/False)

True False

6. Which of the following is NOT one of the five key characteristics of big data (the 5 V's)?

Volume Variety Validity Velocity

7. In the context of data types, temperature measured in degrees Fahrenheit is an example of:

Nominal Data Ordinal Data Interval Data Ratio Data

8. Which of the following best describes a 'data mart'?

A centralized repository for all data across an enterprise. A subset of a data warehouse focused on a specific business line or team. A collection of unstructured data stored in its raw form. An open-source framework for distributed storage and processing of big data.

9. Which of the following is a challenge associated with big data privacy concerns?

Difficulty in finding information due to information overload Extensive profiling and user tracking without explicit consent Legal penalties for non-compliance with regulations Integration complexity of diverse data sources

10. What is the primary function of the 'Transform' stage in the ETL process?

Extracting data from various sources Loading data into the data warehouse Converting data into a suitable format for analysis Deleting obsolete data

11. Explain how Airbnb's data management strategy, particularly the development of the Dataportal, contributed to its rapid growth.

12. Describe the Extract, Transform, Load (ETL) process and explain its importance in populating data warehouses.

13. Explain how Hadoop enables the processing of large data sets and discuss the roles of HDFS and MapReduce in this framework.

1. How can you filter the data.frame nyc_payroll_new to calculate descriptive statistics (mean and standard deviation) of Base_Salary for workers in the Work_Location_Borough "MANHATTAN"? Similarly, how can you filter the data.frame nyc_payroll_new to calculate these statistics for workers in the Work_Location_Borough "QUEENS"?

Provide the R code for performing these calculations and then report the mean and standard deviation of Base_Salary for workers in both "MANHATTAN" and "QUEENS".

2. How can you filter the data.frame nyc_payroll_new to show only the records where the Base_Salary is greater than or equal to $100,000?

Complete the code by filling in the blank:

nyc_payroll_new |>
  filter(__BLANK__)

3. How can you select only distinct combinations of Agency_Name and Title_Description?

Complete the code by filling in the blank:

nyc_payroll_new |> 
  __BLANK__

4. How would you arrange the data by Regular_Gross_Paid in descending order, showing the highest paid employees first?

Complete the code by filling in the blank:

nyc_payroll_new |> 
  __BLANK__

5. How can you select and rename the Title_Description variable to Title?

Complete the code by filling in the blank:

nyc_payroll_new |> 
  rename(__BLANK__)

6. How can you filter the data to show only records for the “POLICE DEPARTMENT” Agency_Name and arrange it by Total_OT_Paid in ascending order?

Complete the code by filling in the blanks:

nyc_payroll_new |> 
  filter(__BLANK 1__) |> 
  arrange(__BLANK 2__)

7. How can you filter the data to include only those records where the Pay_Basis is “per Annum” and then select only the First_Name, Last_Name, and Base_Salary variables?

Complete the code by filling in the blanks:

nyc_payroll_new |> 
  filter(__BLANK 1__) |> 
  select(__BLANK 2__, __BLANK 3__, __BLANK 4__)

8. How would you arrange the data.frame by Work_Location_Borough in ascending order and Base_Salary in descending order?

Complete the code by filling in the blank:

nyc_payroll_new |> 
  __BLANK__

9. How can you filter the nyc_payroll_new data.frame to remove observations where the Base_Salary variable has NA values? After filtering, how would you calculate the total number of remaining observations?

Provide the R code needed to perform these data extraction tasks.

Name, Email, and Session

Short-Answer Questions

Data Transformation with R `tidyverse`

Name, Email, and Session

Short-Answer Questions

Data Transformation with R tidyverse

Data Transformation with R `tidyverse`