dime-worldbank/big-data-poverty-estimation

TODOS

Closed this issue · 1 comments

General TODOS:

  • Follow Yeh's code for getting TFrecords // adapt for sentinel
  • Standardize NTL bands
  • OSM - Extract data (check other paper. try angles? -- (1) cardinal directions, (2) absolute value equivalent; 90 = 270, (3) standard deviation... but up and sideways may still have high SD?). Could do how compares to larger/surrounding area.
  • Get Keras working for individual npys [LINK]. Multiple inputs (stack link); more useful stack here
  • For prepping CNN, add skip if already processed
  • Transfer learning dataset -- beyond just survey locations
  • Can use use larger than 224 for VGG16? If not, what to do about sentinel (survey 4 locations?). Also some sentinel bands aren't 10m
  • Facebook data - if didn't scrape all params, delete
  • DHS: globally consistent asset index (check what other paper does - Yeh et al)
  • Add CNN for NTL
  • Check RGB npy - visualize, should look as expected

ML Model TODOS:

  • Grab actual values from tfrecords (so can compare with predicted)
  • Standardize tfrecord values

Include all bands, including NTL, in one numpy array. Can then grab each one during data generation! Also include wealth vars? Hmmm not needed if named. Put each in country-year folder.