Show trends on maps as part of a term paper.
- Formulate project proposal
- Define scope of project
Requirements
- OSX or Linux
- Python 2.7, Python 3.4 or PyPy
- GNU parallel
Install Python requirements.
pip install -r requirements.txt
You have several possiblities to prepare the data:
- Follow the instructions
- Use
./prepare.sh
- Download prepared tiles directly
Download all logs from the server.
wget -nH -A xz -m http://planet.openstreetmap.org/tile_logs
Extract logs and remove compressed versions.
unxz *.xz
rm *.xz
Expand tile coordinates in CSV to separate columns (from 1/2/3 123
to 1 2 3 123
).
sed -i 's/\// /g' *.txt
Rename .txt
files to .csv
.
rename 's/\.txt$/\.csv/' *.txt
Convert tile indizes from the logs with slippy tile names to Spherical Mercator coordinates add CSV header.
mkdir -p tile_coords
for filename in $(ls tile_logs/*.csv); do
echo $(basename $filename)
echo "z x y requests latitude longitude" > tile_coords/$(basename "$filename")
cat $filename | ./calc_coords.py >> tiles_coords/$(basename "$filename")
done
Convert from space delimited to comma delimited CSV (requirements for use in Mapbox).
mkdir -p tile_mapbox
for filename in $(ls tile_coords/*.csv); do
cat $filename | tr ' ' ',' > tile_mapbox/$(basename "$filename").mapbox.csv
done
aggregate_requests.py
takes in a list of absolute filepaths and sums
up the requests of the tiles.
The csv files must already be prepared with the prepare.sh
script.
ls -d -1 tile_logs/tiles-2015*.csv | ./aggregate_requests.py >> tiles-2015.csv
cat tiles-2015-01-01.csv | awk '$1 == "18"' > cat tiles_18-2015-01-01.csv
echo "date requests latitude longitude" > tiles_over_time_switzerland.csv
for filename in $(ls tile_coords/*.csv); do
base_filename=$(basename "$filename")
date=$(echo "$base_filename" | sed 's/tiles-//' | sed 's/\.csv//')
echo $date
cat $filename | \
awk '$1 == "18"' | \
cut -d ' ' -f4-6 | \
python filter_switzerland.py | \
sed -e "s/^/$date /" >> tiles_over_time_switzerland.csv
done