/read-PME-80s

Read "Pesquisa Mensal do Emprego" 1984-1990

Primary LanguageR

Construct PME datasets

The code here is responsible for converting PME 1984-1990 from its raw form – as distributed by IBGE – into readable csv files.

The two main types of ingredients are the following:

  1. PME datasets, downloaded and treated (see Usage)
  2. PME dictionaries in proper format
    • This repo contains dictionaries that reflect my attempt to parse weird PME documentation files
    • there might be mistakes, I strongly suggest users sift through the files to ensure consistency for their use case

Dependencies

Make sure R is installed, as well as the following packages:

  • data.table
  • glue
  • optparse

Usage

  1. Download PME 1980 to 1990 from IBGE website:

    ftp://ftp.ibge.gov.br/Trabalho_e_Rendimento/Pesquisa_Mensal_de_Emprego/Microdados/1980_2001/PME_1980_a_2001.zip

  2. Run detox utility on all files, 
    detox -r -v $PME_DOWNLOADED_PATH

    where $PME_DOWNLOADED_PATH should be replaced with its actual download path

  3. Clone / download this repo anywhere in your pc
  4. Give write permission to pmeread.r, navigating to the directory and running chmod +x pmeread.r
  5. Run 
    ./pmeread.r -i $PME_DOWNLOADED_PATH

    see ./pmeread.r -h for more options

Tasks

Document

  • [ ] how should datasets be layed out on base_data_path?
  • [ ] how should dictionaries be organized?