duncantl/STA141B_S24

STA141B in Spring 24

STA141B Spring 2024

Instructor: Duncan Temple Lang

I will put code, notes, R sessions from live coding in class, etc. from the course in this github repository and some also on Canvas.

• Piazza

• Canvas

• Bonus Idiom

  • cumsum() and grouping lines
  • Related to and might be useful for assignment 2 - Spam Email.

• Note on Dates & Locales

  • Function

• Note on Character Encoding

  • foo, foo1, foo2, foo3

• New Note on Character Encoding and Content-Type

• Brief Git Tutorial

• Lectures

  • Day 1

  • Day 2

  • Day 2

    • Day 3 R session
    • wea/readWeather function
    • functions for reading a table from the .stat file
    • Outline

  • Day 4

    • Guidelines for writing report
    • lapply() & do.call()
    • Day 3 R session

  • Day 5

    • Scenarios (slides)
    • Regular Expression Language
    • job post 1
    • job post 2

  • Day 6

    • Reading HTTP/Web server log files
      • Support functions
    • Wifi/Mannheim data
    • Familiar, small examples of regular expressions

  • Day 7

    • Weblog Regexp explanation figure
    • Familiar, small examples of regular expressions
    • R session/code

  • Day 8

    • R session

  • Day 9 README

  • Day 10

    • R session
    • Notes on R Code
    • Reading Wifi data

  • Day 11 README

  • Day 12

    • Slides on SQL
    • moz_cookies/cookies3.slite DB
    • HAVING example and exploration
    • R session

  • Day 13

    • slides
      • we started on page 25.
    • R session

  • Day 14

    • Slides
    • SQL session
    • helper functions

  • Day 15

    • Slides
    • Description of HTML tree.
    • NYTimes map example
      • Data in CSV via separate background download
      • NYT.R
    • Marine Traffic
      • Data in JSON via separate background download
      • Can't make simple HTTP request with readLines()
    • Stats StackExchange
      • static HTML content in Question summary front page
      • Get links to page for each question.
    • Firefox Developer Tools
    • Rsession

  • Day 16

    • XPath slides
    • Using selenium to remote-control a Web browser
    • Scraping stats.stackexchange.com & XPath
    • Rsession

  • Day 17

    • code to process search results/pages of question from stats.stackexchange.com
      • This is a good structure for harvesting posts/questions/etc. when we have page after page of search results. In other words, consider using this structure for assignment 4, and specializing it to craigslist. The components correspond to
        • loop over pages and append the results
        • process a page of results
        • process each result, e.g., get URL for actual post/question
        • get URL or HTML for the next page of results
        • fix/post-process the columns in the overall data.frame

  • Day 18

    • GapMinder visualization
    • plotly example
      • R code
      • HTML of plotly example
    • book chapter & plotly examples
    • plotly book
    • animated map & time series
      • runnable version here
    • D3 gallery

  • Day 19

    • Outline
    • plotly output
    • button and "dynamic" HTML tables
    • R-generated plotly plot with event handler - custom data
    • list as custom data

  • Day 20

    • Direct/Low-level Approach
      • Discusses generating the SVG directly from R using R graphics (including ggplot2) and then annotating them from R using XPath and XML manipulation.
    • Annotating SVG plots - succinct
    • Stand-alone version of Animated Map
      • This directly inlines the SVG documents in the HTML and avoids the cross-origin security issue which we dealt with previously by serving the files via a Web server.
    • Annotating ggplot plots