Modify convert.py to translate assessor data into OSM tags
Closed this issue · 12 comments
See #3 for more info about assessor data.
convert.py already combines addresses with buildings using the AIN field. We plan to use the AIN field to join the assessor data too.
Just joined the assessor data straight to the building shapefile. Rehosted at latimes.
Great! I assume we still need to figure out how to translate those assessor fields into OSM tags. Maybe that discussion should go in #3, or maybe it should happen in this issue?
@cityhubla and I did some of that work already. I think the decision was spelled out in the readme. I modified convert.py to handle those. Though, it doesn't line up perfectly with how OSM does things. For instance, OSM community would prefer "building=apartments" instead of "building=Residential"
@almccon do we need to go back to that and redo some of those (in the readme)?
Yes, we should re-run the sample data. In #21 and few other issues, @maning is looking at my sample .osm files, but now they're obsolete since you've incorporated the assessor data.
Unfortunately I don't have time to regenerate those files myself... If anyone else gets a chance to regenerate the samples, please replace the https://github.com/osmlab/labuildings/blob/master/venice_osm.zip file, or change the readme to link to something else.
I regenerated a new sample data here based of the joined dataset from assesor data. I also fixed a few lines in convert.py to handle NULL values in some fields.
We also need to agree on how to convert the SpecificUseType and SpecificUseDetail1 into proper OSM tags. I can do the attribute matching. Should I, convert to OSM tags into the source shapefile or should we create another script to clean this up (former should be easier for me)?
@maning how would you go about converting the OSM tags in the source shapefile? I'm guessing some scripting logic?
Going through the attributes spreadsheet, an important question for me is if it is sensible to import so much street detail from 10 years ago without a name=* tag? it would be a massive task for mappers to verify and clean this data once it is imported into OSM.
The SpecificUseType which makes sense for me to import are those that are verifiable from imagery to some extent:
- parking
- fuel station
- school
- college
- hospital
- golf course
Otherwise we should just stick to importing the building=* value best we can and punt on others till we can get a more updated dataset rather than a huge conflation/verification headache later on.
@jschleuss, per conversation with my team, it make sense to process the tag through a "sanitizer" script before writing the xml file. Depending on the implementation, it can either be separate script where we feed a csv of OSM tags based on the spreadsheet or expand convert.py to do this similar to what it does for addresses.
Before implementation, we need to agree first if its worth the effort to do this considering the above comment by @planemad
@planemad @maning I'm not sure what data is 10 years old. Do you mean the building shapefile and AINs from the 2008 file released in 2009? The Assessor just recently released their 2015 roll information and that's what we've attached to the building file. So the data attributes should be pretty fresh. I don't see a compelling reason to drop it, but would see the need to write the sanitizing script to better match the OSM tags.
@jschleuss thanks for clarifying, I missed the fact that the assessor data was released in 2015, so that makes more sense to import.
How will we handle amenity/shop tags on building parts which have the SpecificUseType field? #19 (comment)
Closing.