fabiogiglietto/CooRnet

Example CT posts/search code?

networks1 opened this issue · 2 comments

networks1 commented

You guys used to offer a worked example of how to run CooRNet via a CT posts/search call. The current code example assumes I'm seeding the search with links from from MediaCloud, which is not what I want to do. Is the posts/search functionality gone? If not can you please include a brief example of how to execute that?

fabiogiglietto commented

Hi Steve,
you have two options here:

  1. Getting a search or historical CSV from CrowdTangle web interface and doing something like this to get your urls:
library("readr")
library("CooRnet")

# setup working spaces with data and rawdata folders
dir.create("./data/")
dir.create("./rawdata/")

# set the link to CrowdTangle CSV file\

allpostsfile <- "LINK-TO-CSV"

# un-comment the following lines if you want to inspect the CSV file
# df <- read.csv(allpostsfile)
# names(df)
# head(df)

urls <- CooRnet::get_urls_from_ct_histdata(ct_histdata_csv = allpostsfile,
                                           newformat = TRUE)

nrow(urls) # get the number of URLs
names(urls) # list the columns name
write.csv(urls, "./data/urls.csv") # save the list of URLs on disk
  1. As you pointed out, you can get the post to extract the links from by directly querying the posts/search endpoint of the API (this is great for automation). I don't have a code example at hand but you can start from this function to adapt the code to extract links from a set of posts retrieved via posts/search.

Hope it helps! Best, Fabio

networks1 commented