Complete Guide to Programming Automation with R and RStudio

It is time to learn about packages in R and install some very useful ones necessary for this course.

Attached is a script which can help to install all packages at once.

Note for MAC OS

Please run brew install poppler before installing the package pdftools

• Overview of RStudio and Folder Navigation

The lecture covers basics functions that list files in folders, retrieve names, paths and perform sanity checks.

In this lecture we learn how to write simple loops to read all files and aggregate data:

• For loop with itterations

• Bind data by rows

In this lecture we learn how to work with data frames:

• Remove duplicate lines

• Extract information on content, type, data structures

• Use pipe operator for clean code

• Data type conversions (from String to Date)

• Add or edit column data in a data frame

In this lecture we learn how to Analyze, Subset and Write Data:

• Count by categories

• Subset and filter by categories or by values

• Create Folders

• Write CSV files

In this lecture we learn how to Extract and Calculate Metrics from Data:

• Subset by year/month

• Pull values from a column

• Compute sum, mean, median, max, min

In this lecture we cover in detail the csv and txt formats, encodings, arguments to the functions.

In this lecture we learn how to read XLSX files:

• Load Workbook and retrive Information

• Load Data from Sheets

  
  

In this lesson we learn how to parse PDF files:

• Parse content as text

• Parse text using regex tools

In this lecture we learn how to Download Files from Web:

• Download from URL

• Read directly from URL

In this lecture we learn how to scrap tables from public webpages without any complex html tools.

In this lecture we learn how to read public files from Cloud locations:

• Read open Google Sheets

• Read open Dropbox Files

For private files please see the Extra lessons.

In this lecture we learn how to configure the connection to the Email box:

• IMAP configuration

• Gmail security settings

• List folders of an Email box

In this lecture we learn how to search emails:

• Filtering by dates

• Filtering by sender, recipients, keywords

In this lecture we learn how to manipulate the content of an email with Text Manipulation Tools

• Retrive content using Text Patterns

• Extract Values with String Manipulation functions

In this lecture we learn how to fetch the attachments from an Email.

In this lecture we learn how to configure the connection to an Email box.

In this lecture we learn how to create emails and send them:

• Create Body and Title

• Add styles to Body with HTML

• Add attachments

In this lecture we create a complex project in which we cover the skills we learned previously:

• Create a Config file

• Setup the Script

In this lecture we read and aggregate the data.

We also learn how to manipulate Date objects in R.

In this lecture we learn how to create a custom email:

• Automatic subject text

• Add text content to body

• Add HTML table with CSS formatting

In this lecture we learn how to work with XLSX files:

• Load or Create Excel Workbook

• Save the files as XLSX

• Write basic Data

In this lecture we learn how to Style content in a XLSX file:

• Custom position of a Table

• Format of Numbers and Dates

• Custom Borders

• Cell Font Size, Colors, Background, Alignment

• Apply styles to specific Cells

• Cell Height, Width

In this lecture we learn the Conditional Formatting feature:

• Rule based on Value

• Databar with custom Color

In this lecture we learn how to Create Excel with Multiple Pages and we go through some functions that aren't covered in details in this course.

• Loop and Backward Loop

• Sheet Visibility

• File Paths

Extra functions.

In this lecture we learn how to create Powerpoint documents:

• Read and Write Slide

• Add slide with Content

• Temporary Files

  
  

In this lecture we learn how to create nicely Formatted Content in our PowerPoint:

• Add Title and Body Text

• Format Text: Font, Size, Color

• Paragraphs Formatting

• Add Default Table

• Add Flextable in Body

In this lecture we learn how to create nicely Formatted Content in our Word:

• Format Text: Font, Size, Color

• Paragraphs Formatting

• Add Flextable in Body

• Add Multiple Pages

In this lecture we learn how to use Github and Git:

• Create an Account

• Create a Token

• Install Git and connect to RStudio

• Create a Repository

  
  

In this lecture we learn how to manage a Repository with Code:

• Clone Repo to RStudio

• Add Code, Commit and Push

In this lecture we learn how to use Dropbox as a Cloud Data source:

• Connect Account and Create Token

• List Files and Download Locally

• Upload Files from Local Drive

  
  

In this lecture we aggregate previously learned skills and create a complex report with Email Body, Attachments and Dropbox as Data Source.

In this lecture we learn how to Schedule Tasks on a Windows machine:

• Configure Scripts

• Schedule Timed Tasks

• Windows Options

In this lecture we learn how to work with Amazon Web Services:

• Region Configuration

• Free Tier Solutions

• Overview of Services

• Putty Software https://www.putty.org/

In this lecture we learn how to create a EC2 Instance:

• Ubuntu Server

• Overview of Instance Types

• Configure New Instance

• Create .pem Key

• Convert Key using Puttyy

• Connect using SSH

• Install R Demo (optimal way covered in next lesson)

In this lecture we learn how to Configure R on a Server Automatically:

• Install Ubuntu and R Libraries on Launch

In this lecture we learn how to work with the Linux Terminal:

• Run R and R commands

• Linux Commands to Navigate through Folders and Files

• Clone Repo from Github

• Run Rscripts

• Update Folder with Git Pull

In this lecture we learn how to run complex script on the Server:

• Install a Package Manually on Linux

• Repository For Linux

• Gmail Security on Linux

• Configure Correctly the Paths to Folders

• Amend Script and Push-Pull

In this lecture we learn how to schedule a CRON (scheduled repetitive tasks) in Linux:

• Configure path to home

• Create CRON Job

• Configure schedule using crontab.guru

• Task in Crontab

• Linux Commands

• Read Logs