Add "summary" section to schema
Closed this issue · 10 comments
tomschenkjr commented
Users of the building.json file will need to look at some summary statistics. We should consider a summary section of the file in the schema.
...
"summary": {
911: {
"last_7_days_total": 6,
"last_14_days_total": 9,
"month_over_month_change_total": -5,
"month_over_month_change_perc": -0.14
},
311: {
...
}
}
...
As a related question, we need to determine which summary measures should be included. For the sake of later application development, each one of summary measures should be the same in each subgroup.
nicklucius commented
There is still some organization to do, but the code is set to do the calculations and proper nesting for the summary data. Here is an example record. The nested nature makes the organization work a bit more involved.
{
"address": "4517 N MONTICELLO AVE"
"bldg_id": "121238",
"pins": ["1314120011"],
"area": [156.1104],
"areaRatio": [1],
"pinsFinal": ["1314120011"],
"annual_sale.alltime": [2],
"annual_sale.pastyear": [1],
"annual_sale.past90days": [0],
"annual_sale.past30days": [0],
"annual_sale": [
{
"buyer_name": "Scribe Funding",
"classification": "203",
"cost": 200,
"penalty_amount_offered": 352.66,
"pin": "1314120011",
"sold_at_sale": "TRUE",
"tax_amount_offered": 2137.32,
"tax_sale_year": "2014",
"total_amount_paid": 2689.98,
"total_tax_and_penalty_amount_offered": 2489.98,
"township_name": "JEFFERSON",
"winning_bid_percent": "0",
"location_1.type": "Point",
"location_1.coordinates": [-87.719, 41.9635],
"feature_date": "2017-08-23"
},
{
"buyer_name": "Interstate Funding Corp.",
"classification": "203",
"cost": 200,
"penalty_amount_offered": 1089.38,
"pin": "1314120011",
"sold_at_sale": "TRUE",
"tax_amount_offered": 4630.86,
"tax_sale_year": "2011",
"total_amount_paid": 5920.24,
"total_tax_and_penalty_amount_offered": 5720.24,
"township_name": "JEFFERSON",
"winning_bid_percent": "0",
"location_1.type": "Point",
"location_1.coordinates": [-87.719, 41.9635],
"feature_date": "2014-08-23"
}
],
"scavenger_sale.alltime": [0],
"scavenger_sale.pastyear": [0],
"scavenger_sale.past90days": [0],
"scavenger_sale.past30days": [0],
"scavenger_sale": [],
"demoBldg.alltime": [7],
"demoBldg.pastyear": [0],
"demoBldg.past90days": [0],
"demoBldg.past30days": [0],
"demoBldg": [
{
"HOUSE_LOW": 4517,
"PRE": "N",
"STREET_NAME": "MONTICELLO",
"STREET_TYPE": "AVE",
"ID": 4800348,
"VIOLATION.LAST.MODIFIED.DATE": "07/21/2014",
"VIOLATION.DATE": "2014/07/21",
"VIOLATION.CODE": "CN002081",
"VIOLATION.STATUS": "OPEN",
"VIOLATION.STATUS.DATE": "",
"VIOLATION.DESCRIPTION": "SUBMIT FOR APPROVAL PLANS",
"VIOLATION.LOCATION": "",
"VIOLATION.INSPECTOR.COMMENTS": "",
"VIOLATION.ORDINANCE": "Submit for approval plans prepared, signed, and sealed by licensed architect or licensed engineer and obtain permit for all but emergency repairs. (13-32-010, 13-40-010, 13-40-020)",
"INSPECTOR.ID": "BL00704",
"INSPECTION.NUMBER": 11297229,
"INSPECTION.STATUS": "CLOSED",
"INSPECTION.WAIVED": "N",
"INSPECTION.CATEGORY": "COMPLAINT",
"DEPARTMENT.BUREAU": "DEMOLITION",
"ADDRESS": "4517 N MONTICELLO AVE",
"PROPERTY.GROUP": 178559,
"FULL_ADDR": "4517 N MONTICELLO AVE",
"HOUSE_HIGH": 4517,
"SUFDIR": " ",
"XCOORD": 1151280.2675,
"YCOORD": 1929853.0994,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7192,
"latitude": 41.9634,
"bldg_id": "121238",
"feature_date": "2014-07-21"
},
{
"HOUSE_LOW": 4517,
"PRE": "N",
"STREET_NAME": "MONTICELLO",
"STREET_TYPE": "AVE",
"ID": 4800349,
"VIOLATION.LAST.MODIFIED.DATE": "07/21/2014",
"VIOLATION.DATE": "2014/07/21",
"VIOLATION.CODE": "CN193000",
"VIOLATION.STATUS": "OPEN",
"VIOLATION.STATUS.DATE": "",
"VIOLATION.DESCRIPTION": "MAINTAN BLDG IN SAFE CONDITION",
"VIOLATION.LOCATION": "",
"VIOLATION.INSPECTOR.COMMENTS": "",
"VIOLATION.ORDINANCE": "Maintain building in safe condition so it does not constitute actual and imminent danger to public. (13-12-130, 13-8-100, 13-12-120, 13-12-125, 65 ILCS 5/11-31-1) Maintain building windows and doors in safe condition. (13-196-550) Maintain stairway and porch in safe condition. (13-196-570) Maintain floors, walls, and ceilings in safe condition. (13-196-540) Maintain building in safe condition. Premises now dangerous as vacant and open, uncompleted and abandoned, or vacant and boarded. (13-12-130, 13-12-125) Maintain watchman at vacant or open premises. (13-12-140)",
"INSPECTOR.ID": "BL00704",
"INSPECTION.NUMBER": 11297229,
"INSPECTION.STATUS": "CLOSED",
"INSPECTION.WAIVED": "N",
"INSPECTION.CATEGORY": "COMPLAINT",
"DEPARTMENT.BUREAU": "DEMOLITION",
"ADDRESS": "4517 N MONTICELLO AVE",
"PROPERTY.GROUP": 178559,
"FULL_ADDR": "4517 N MONTICELLO AVE",
"HOUSE_HIGH": 4517,
"SUFDIR": " ",
"XCOORD": 1151280.2675,
"YCOORD": 1929853.0994,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7192,
"latitude": 41.9634,
"bldg_id": "121238",
"feature_date": "2014-07-21"
},
{
"HOUSE_LOW": 4517,
"PRE": "N",
"STREET_NAME": "MONTICELLO",
"STREET_TYPE": "AVE",
"ID": 4800350,
"VIOLATION.LAST.MODIFIED.DATE": "07/21/2014",
"VIOLATION.DATE": "2014/07/21",
"VIOLATION.CODE": "CN193019",
"VIOLATION.STATUS": "OPEN",
"VIOLATION.STATUS.DATE": "",
"VIOLATION.DESCRIPTION": "REPAIR/WRECK DANGER RESID PREM",
"VIOLATION.LOCATION": "",
"VIOLATION.INSPECTOR.COMMENTS": "",
"VIOLATION.ORDINANCE": "Repair or wreck dangerous and vacant residential premises. (13-8-100, 13-12-125, 13-12-130)",
"INSPECTOR.ID": "BL00704",
"INSPECTION.NUMBER": 11297229,
"INSPECTION.STATUS": "CLOSED",
"INSPECTION.WAIVED": "N",
"INSPECTION.CATEGORY": "COMPLAINT",
"DEPARTMENT.BUREAU": "DEMOLITION",
"ADDRESS": "4517 N MONTICELLO AVE",
"PROPERTY.GROUP": 178559,
"FULL_ADDR": "4517 N MONTICELLO AVE",
"HOUSE_HIGH": 4517,
"SUFDIR": " ",
"XCOORD": 1151280.2675,
"YCOORD": 1929853.0994,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7192,
"latitude": 41.9634,
"bldg_id": "121238",
"feature_date": "2014-07-21"
},
{
"HOUSE_LOW": 4517,
"PRE": "N",
"STREET_NAME": "MONTICELLO",
"STREET_TYPE": "AVE",
"ID": 4800351,
"VIOLATION.LAST.MODIFIED.DATE": "07/21/2014",
"VIOLATION.DATE": "2014/07/21",
"VIOLATION.CODE": "CN193029",
"VIOLATION.STATUS": "OPEN",
"VIOLATION.STATUS.DATE": "",
"VIOLATION.DESCRIPTION": "WATCHMAN",
"VIOLATION.LOCATION": "",
"VIOLATION.INSPECTOR.COMMENTS": "",
"VIOLATION.ORDINANCE": "Maintain watchman from 4:00 PM to 8:00 AM for vacant and dangerous residential premises. (13-12-140)",
"INSPECTOR.ID": "BL00704",
"INSPECTION.NUMBER": 11297229,
"INSPECTION.STATUS": "CLOSED",
"INSPECTION.WAIVED": "N",
"INSPECTION.CATEGORY": "COMPLAINT",
"DEPARTMENT.BUREAU": "DEMOLITION",
"ADDRESS": "4517 N MONTICELLO AVE",
"PROPERTY.GROUP": 178559,
"FULL_ADDR": "4517 N MONTICELLO AVE",
"HOUSE_HIGH": 4517,
"SUFDIR": " ",
"XCOORD": 1151280.2675,
"YCOORD": 1929853.0994,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7192,
"latitude": 41.9634,
"bldg_id": "121238",
"feature_date": "2014-07-21"
},
{
"HOUSE_LOW": 4517,
"PRE": "N",
"STREET_NAME": "MONTICELLO",
"STREET_TYPE": "AVE",
"ID": 4800352,
"VIOLATION.LAST.MODIFIED.DATE": "07/21/2014",
"VIOLATION.DATE": "2014/07/21",
"VIOLATION.CODE": "CN193110",
"VIOLATION.STATUS": "OPEN",
"VIOLATION.STATUS.DATE": "",
"VIOLATION.DESCRIPTION": "VACANT BUILDING - REGISTER",
"VIOLATION.LOCATION": "",
"VIOLATION.INSPECTOR.COMMENTS": "",
"VIOLATION.ORDINANCE": "Register vacant building within 30 days of it becoming vacant, or within 30 days after assuming ownership of an existing vacant building. (13-12-125(a)). Building must be kept in compliance with all vacant building requirements pursuant to 13-12-135. See Vacant Building Ordinance and registration form at https://ipiweb.cityofchicago.org/VBR",
"INSPECTOR.ID": "BL00704",
"INSPECTION.NUMBER": 11297229,
"INSPECTION.STATUS": "CLOSED",
"INSPECTION.WAIVED": "N",
"INSPECTION.CATEGORY": "COMPLAINT",
"DEPARTMENT.BUREAU": "DEMOLITION",
"ADDRESS": "4517 N MONTICELLO AVE",
"PROPERTY.GROUP": 178559,
"FULL_ADDR": "4517 N MONTICELLO AVE",
"HOUSE_HIGH": 4517,
"SUFDIR": " ",
"XCOORD": 1151280.2675,
"YCOORD": 1929853.0994,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7192,
"latitude": 41.9634,
"bldg_id": "121238",
"feature_date": "2014-07-21"
},
{
"HOUSE_LOW": 4517,
"PRE": "N",
"STREET_NAME": "MONTICELLO",
"STREET_TYPE": "AVE",
"ID": 4800353,
"VIOLATION.LAST.MODIFIED.DATE": "07/21/2014",
"VIOLATION.DATE": "2014/07/21",
"VIOLATION.CODE": "CN196039",
"VIOLATION.STATUS": "OPEN",
"VIOLATION.STATUS.DATE": "",
"VIOLATION.DESCRIPTION": "POST OWNER INFO ON ABDN BLDG",
"VIOLATION.LOCATION": "",
"VIOLATION.INSPECTOR.COMMENTS": "",
"VIOLATION.ORDINANCE": "Post, on abandoned building, name, address and telephone of owner and owner's agent for managing, controlling or collecting rents on building so legible from nearest public street or sidewalk. (13-12-125)",
"INSPECTOR.ID": "BL00704",
"INSPECTION.NUMBER": 11297229,
"INSPECTION.STATUS": "CLOSED",
"INSPECTION.WAIVED": "N",
"INSPECTION.CATEGORY": "COMPLAINT",
"DEPARTMENT.BUREAU": "DEMOLITION",
"ADDRESS": "4517 N MONTICELLO AVE",
"PROPERTY.GROUP": 178559,
"FULL_ADDR": "4517 N MONTICELLO AVE",
"HOUSE_HIGH": 4517,
"SUFDIR": " ",
"XCOORD": 1151280.2675,
"YCOORD": 1929853.0994,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7192,
"latitude": 41.9634,
"bldg_id": "121238",
"feature_date": "2014-07-21"
},
{
"HOUSE_LOW": 4517,
"PRE": "N",
"STREET_NAME": "MONTICELLO",
"STREET_TYPE": "AVE",
"ID": 4907395,
"VIOLATION.LAST.MODIFIED.DATE": "05/17/2016",
"VIOLATION.DATE": "2014/11/12",
"VIOLATION.CODE": "CN193039",
"VIOLATION.STATUS": "COMPLIED",
"VIOLATION.STATUS.DATE": "11/12/2014",
"VIOLATION.DESCRIPTION": "POST OWNER NAME OF VACNT BLDG",
"VIOLATION.LOCATION": "",
"VIOLATION.INSPECTOR.COMMENTS": "VIOLATION CLOSED PER PCR 1180",
"VIOLATION.ORDINANCE": "Post conspicuously name, address, and telephone of owner and manager of vacant residential premises. (13-12-125).",
"INSPECTOR.ID": "BL00900",
"INSPECTION.NUMBER": 11312937,
"INSPECTION.STATUS": "CLOSED",
"INSPECTION.WAIVED": "N",
"INSPECTION.CATEGORY": "COMPLAINT",
"DEPARTMENT.BUREAU": "DEMOLITION",
"ADDRESS": "4517 N MONTICELLO AVE",
"PROPERTY.GROUP": 178559,
"FULL_ADDR": "4517 N MONTICELLO AVE",
"HOUSE_HIGH": 4517,
"SUFDIR": " ",
"XCOORD": 1151280.2675,
"YCOORD": 1929853.0994,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7192,
"latitude": 41.9634,
"bldg_id": "121238",
"feature_date": "2014-11-12"
}
],
"demoPIN.alltime": [0],
"demoPIN.pastyear": [0],
"demoPIN.past90days": [0],
"demoPIN.past30days": [0],
"demoPIN": [],
"vacant311Bldg.alltime": 2,
"vacant311Bldg.pastyear": 0,
"vacant311Bldg.past90days": 0,
"vacant311Bldg.past30days": 0,
"vacant311Bldg": [
{
"ADDRESS.STREET.NUMBER": "4517",
"ADDRESS.STREET.DIRECTION": "N",
"ADDRESS.STREET.NAME": "MONTICELLO",
"ADDRESS.STREET.SUFFIX": "AVE",
"SERVICE.REQUEST.TYPE": "Vacant/Abandoned Building",
"SERVICE.REQUEST.NUMBER": "14-01162277",
"DATE.SERVICE.REQUEST.WAS.RECEIVED": "2014/07/21",
"LOCATION.OF.BUILDING.ON.THE.LOT..IF.GARAGE..CHANGE.TYPE.CODE.TO.BGD..": "Front",
"IS.THE.BUILDING.DANGEROUS.OR.HAZARDOUS.": "",
"IS.BUILDING.OPEN.OR.BOARDED.": "Open",
"IF.THE.BUILDING.IS.OPEN..WHERE.IS.THE.ENTRY.POINT.": "",
"IS.THE.BUILDING.CURRENTLY.VACANT.OR.OCCUPIED.": "Vacant",
"IS.THE.BUILDING.VACANT.DUE.TO.FIRE.": "false",
"ANY.PEOPLE.USING.PROPERTY...HOMELESS..CHILDEN..GANGS.": "false",
"ZIP.CODE": 60625,
"Ward": 33,
"Police.District": 17,
"Community.Area": 14,
"FULL_ADDR": "4517 N MONTICELLO AVE",
"HOUSE_LOW": 4517,
"HOUSE_HIGH": 4517,
"PRE": "N",
"STREET_NAME": "MONTICELLO",
"STREET_TYPE": "AVE",
"SUFDIR": " ",
"XCOORD": 1151280.2675,
"YCOORD": 1929853.0994,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7192,
"latitude": 41.9634,
"bldg_id": "121238",
"feature_date": "2014-07-21"
},
{
"ADDRESS.STREET.NUMBER": "4517",
"ADDRESS.STREET.DIRECTION": "N",
"ADDRESS.STREET.NAME": "MONTICELLO",
"ADDRESS.STREET.SUFFIX": "AVE",
"SERVICE.REQUEST.TYPE": "Vacant/Abandoned Building",
"SERVICE.REQUEST.NUMBER": "14-01399048",
"DATE.SERVICE.REQUEST.WAS.RECEIVED": "2014/08/23",
"LOCATION.OF.BUILDING.ON.THE.LOT..IF.GARAGE..CHANGE.TYPE.CODE.TO.BGD..": "Front",
"IS.THE.BUILDING.DANGEROUS.OR.HAZARDOUS.": "",
"IS.BUILDING.OPEN.OR.BOARDED.": "Open",
"IF.THE.BUILDING.IS.OPEN..WHERE.IS.THE.ENTRY.POINT.": "FRONT",
"IS.THE.BUILDING.CURRENTLY.VACANT.OR.OCCUPIED.": "Vacant",
"IS.THE.BUILDING.VACANT.DUE.TO.FIRE.": "false",
"ANY.PEOPLE.USING.PROPERTY...HOMELESS..CHILDEN..GANGS.": "true",
"ZIP.CODE": 60625,
"Ward": 33,
"Police.District": 17,
"Community.Area": 14,
"FULL_ADDR": "4517 N MONTICELLO AVE",
"HOUSE_LOW": 4517,
"HOUSE_HIGH": 4517,
"PRE": "N",
"STREET_NAME": "MONTICELLO",
"STREET_TYPE": "AVE",
"SUFDIR": " ",
"XCOORD": 1151280.2675,
"YCOORD": 1929853.0994,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7192,
"latitude": 41.9634,
"bldg_id": "121238",
"feature_date": "2014-08-23"
}
],
"vacant311PIN.alltime": [0],
"vacant311PIN.pastyear": 0,
"vacant311PIN.past90days": [0],
"vacant311PIN.past30days": 0,
"vacant311PIN": [],
"no_stories": "0",
"suf_dir1": "",
"x_coord": "1151323.31904724",
"shape_area": "1680.375",
"year_built": "1916",
"cdb_city_i": "",
"stories": "1",
"y_coord": "1929879.8931579",
"unit_name": "",
"qc_date": "2003/06/16 07:00:00+00",
"non_standa": "",
"label_hous": "4517",
"st_name1": "MONTICELLO",
"t_add1": "4517",
"z_coord": "0",
"qc_userid": "DS06284",
"qc_source": "PARCELS",
"orig_bldg_": "121238",
"footprint_": "AERIALS98",
"st_type1": "AVE",
"bldg_statu": "ACTIVE",
"condition_": "2003/03/01 08:00:00+00",
"bldg_name2": "",
"bldg_activ": "1998/04/01 08:00:00+00",
"edit_useri": "",
"bldg_sq_fo": "1024.0",
"edit_sourc": "",
"f_add1": "4517",
"vacancy_st": "",
"bldg_creat": "1998/04/01 08:00:00+00",
"no_of_unit": "1",
"create_use": "",
"pre_dir1": "N",
"bldg_condi": "SOUND",
"bldg_name1": "",
"comments": "",
"shape_len": "183.783330346",
"harris_str": "01314120011000",
}
nicklucius commented
I've worked in the nesting. Some of the aggregations in the summary still need to be combined further. For example, some metrics are still separated by whether the records were linked by the building or linked by the pin. This distinction should not be preserved in the summary.
{
"Address": "3836 W JACKSON BLVD",
"Coordinates": {
"x_coord": ["1150691.26092129"],
"y_coord": ["1898494.26044185"]
},
"Summary": {
"TaxSales": [
{
"annual_sale.alltime": [2],
"annual_sale.pastyear": [1],
"annual_sale.past90days": [0],
"annual_sale.past30days": [0],
"scavenger_sale.alltime": [0],
"scavenger_sale.pastyear": [0],
"scavenger_sale.past90days": [0],
"scavenger_sale.past30days": [0]
}
],
"VacancyReports": [
{
"vacant311Bldg.alltime": 2,
"vacant311Bldg.pastyear": 0,
"vacant311Bldg.past90days": 0,
"vacant311Bldg.past30days": 0,
"vacant311PIN.alltime": 0,
"vacant311PIN.pastyear": 0,
"vacant311PIN.past90days": 0,
"vacant311PIN.past30days": 0
}
],
"DemolitionViolations": [
{
"demoBldg.alltime": [7],
"demoBldg.pastyear": [0],
"demoBldg.past90days": [0],
"demoBldg.past30days": [0],
"demoPIN.alltime": [0],
"demoPIN.pastyear": [0],
"demoPIN.past90days": [0],
"demoPIN.past30days": [0]
}
]
},
"Metadata": {
"attributes": [
{
"bldg_id": "361919",
"no_stories": "0",
"suf_dir1": "",
"x_coord": "1150691.26092129",
"shape_area": "1894.875",
"year_built": "1909",
"cdb_city_i": "",
"stories": "2",
"y_coord": "1898494.26044185",
"unit_name": "",
"qc_date": "2003/06/16 07:00:00+00",
"non_standa": "",
"label_hous": "3836",
"st_name1": "JACKSON",
"t_add1": "3836",
"z_coord": "0",
"qc_userid": "DS06284",
"qc_source": "PARCELS",
"orig_bldg_": "361919",
"footprint_": "AERIALS98",
"st_type1": "BLVD",
"bldg_statu": "ACTIVE",
"condition_": "2001/10/01 07:00:00+00",
"bldg_name2": "",
"bldg_activ": "1998/04/01 08:00:00+00",
"edit_useri": "",
"bldg_sq_fo": "3174.0",
"edit_sourc": "",
"f_add1": "3836",
"vacancy_st": "",
"bldg_creat": "1998/04/01 08:00:00+00",
"no_of_unit": "2",
"create_use": "",
"pre_dir1": "W",
"bldg_condi": "NEEDS MINOR REPAIR",
"bldg_name1": "",
"comments": "",
"shape_len": "210.428089649",
"harris_str": "01614107020000",
"pins": ["1614107020"],
"area": [176.038],
"areaRatio": [1],
"pinsFinal": ["1614107020"]
}
]
},
"TaxSales": [
{
"annual_sale": [
{
"buyer_name": "ROYCE RE, LLC",
"classification": "211",
"cost": 200,
"penalty_amount_offered": 298.76,
"pin": "1614107020",
"sold_at_sale": "TRUE",
"tax_amount_offered": 1810.94,
"tax_sale_year": "2014",
"total_amount_paid": 2309.7,
"total_tax_and_penalty_amount_offered": 2109.7,
"township_name": "WEST CHICAGO",
"winning_bid_percent": "0",
"location_1.type": "Point",
"location_1.coordinates": [-87.7222, 41.8774],
"feature_date": "2017-08-29"
},
{
"buyer_name": "MTAG Cust CAZ Creek IL LLC",
"classification": "211",
"cost": 200,
"penalty_amount_offered": 361.01,
"pin": "1614107020",
"sold_at_sale": "TRUE",
"tax_amount_offered": 1851.01,
"tax_sale_year": "2011",
"total_amount_paid": 2412.02,
"total_tax_and_penalty_amount_offered": 2212.02,
"township_name": "WEST CHICAGO",
"winning_bid_percent": "0",
"location_1.type": "Point",
"location_1.coordinates": [-87.7222, 41.8774],
"feature_date": "2014-08-29"
}
],
"scavenger_sale": []
}
],
"DemoViolations": [
{
"demoBldg": [
{
"HOUSE_LOW": 3836,
"PRE": "W",
"STREET_NAME": "JACKSON",
"STREET_TYPE": "BLVD",
"ID": 4709077,
"VIOLATION.LAST.MODIFIED.DATE": "04/14/2014",
"VIOLATION.DATE": "2014/04/14",
"VIOLATION.CODE": "CN002081",
"VIOLATION.STATUS": "OPEN",
"VIOLATION.STATUS.DATE": "",
"VIOLATION.DESCRIPTION": "SUBMIT FOR APPROVAL PLANS",
"VIOLATION.LOCATION": "",
"VIOLATION.INSPECTOR.COMMENTS": "",
"VIOLATION.ORDINANCE": "Submit for approval plans prepared, signed, and sealed by licensed architect or licensed engineer and obtain permit for all but emergency repairs. (13-32-010, 13-40-010, 13-40-020)",
"INSPECTOR.ID": "BL00952",
"INSPECTION.NUMBER": 11233085,
"INSPECTION.STATUS": "CLOSED",
"INSPECTION.WAIVED": "N",
"INSPECTION.CATEGORY": "COMPLAINT",
"DEPARTMENT.BUREAU": "DEMOLITION",
"ADDRESS": "3836 W JACKSON BLVD",
"PROPERTY.GROUP": 248980,
"FULL_ADDR": "3836 W JACKSON BLVD",
"HOUSE_HIGH": 3836,
"SUFDIR": " ",
"XCOORD": 1150686.3596,
"YCOORD": 1898448.7671,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7222,
"latitude": 41.8772,
"bldg_id": "361919",
"feature_date": "2014-04-14"
},
{
"HOUSE_LOW": 3836,
"PRE": "W",
"STREET_NAME": "JACKSON",
"STREET_TYPE": "BLVD",
"ID": 4709078,
"VIOLATION.LAST.MODIFIED.DATE": "04/14/2014",
"VIOLATION.DATE": "2014/04/14",
"VIOLATION.CODE": "CN193000",
"VIOLATION.STATUS": "OPEN",
"VIOLATION.STATUS.DATE": "",
"VIOLATION.DESCRIPTION": "MAINTAN BLDG IN SAFE CONDITION",
"VIOLATION.LOCATION": "",
"VIOLATION.INSPECTOR.COMMENTS": "",
"VIOLATION.ORDINANCE": "Maintain building in safe condition so it does not constitute actual and imminent danger to public. (13-12-130, 13-8-100, 13-12-120, 13-12-125, 65 ILCS 5/11-31-1) Maintain building windows and doors in safe condition. (13-196-550) Maintain stairway and porch in safe condition. (13-196-570) Maintain floors, walls, and ceilings in safe condition. (13-196-540) Maintain building in safe condition. Premises now dangerous as vacant and open, uncompleted and abandoned, or vacant and boarded. (13-12-130, 13-12-125) Maintain watchman at vacant or open premises. (13-12-140)",
"INSPECTOR.ID": "BL00952",
"INSPECTION.NUMBER": 11233085,
"INSPECTION.STATUS": "CLOSED",
"INSPECTION.WAIVED": "N",
"INSPECTION.CATEGORY": "COMPLAINT",
"DEPARTMENT.BUREAU": "DEMOLITION",
"ADDRESS": "3836 W JACKSON BLVD",
"PROPERTY.GROUP": 248980,
"FULL_ADDR": "3836 W JACKSON BLVD",
"HOUSE_HIGH": 3836,
"SUFDIR": " ",
"XCOORD": 1150686.3596,
"YCOORD": 1898448.7671,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7222,
"latitude": 41.8772,
"bldg_id": "361919",
"feature_date": "2014-04-14"
},
{
"HOUSE_LOW": 3836,
"PRE": "W",
"STREET_NAME": "JACKSON",
"STREET_TYPE": "BLVD",
"ID": 4709079,
"VIOLATION.LAST.MODIFIED.DATE": "04/14/2014",
"VIOLATION.DATE": "2014/04/14",
"VIOLATION.CODE": "CN193019",
"VIOLATION.STATUS": "OPEN",
"VIOLATION.STATUS.DATE": "",
"VIOLATION.DESCRIPTION": "REPAIR/WRECK DANGER RESID PREM",
"VIOLATION.LOCATION": "",
"VIOLATION.INSPECTOR.COMMENTS": "",
"VIOLATION.ORDINANCE": "Repair or wreck dangerous and vacant residential premises. (13-8-100, 13-12-125, 13-12-130)",
"INSPECTOR.ID": "BL00952",
"INSPECTION.NUMBER": 11233085,
"INSPECTION.STATUS": "CLOSED",
"INSPECTION.WAIVED": "N",
"INSPECTION.CATEGORY": "COMPLAINT",
"DEPARTMENT.BUREAU": "DEMOLITION",
"ADDRESS": "3836 W JACKSON BLVD",
"PROPERTY.GROUP": 248980,
"FULL_ADDR": "3836 W JACKSON BLVD",
"HOUSE_HIGH": 3836,
"SUFDIR": " ",
"XCOORD": 1150686.3596,
"YCOORD": 1898448.7671,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7222,
"latitude": 41.8772,
"bldg_id": "361919",
"feature_date": "2014-04-14"
},
{
"HOUSE_LOW": 3836,
"PRE": "W",
"STREET_NAME": "JACKSON",
"STREET_TYPE": "BLVD",
"ID": 4709080,
"VIOLATION.LAST.MODIFIED.DATE": "04/14/2014",
"VIOLATION.DATE": "2014/04/14",
"VIOLATION.CODE": "CN193029",
"VIOLATION.STATUS": "OPEN",
"VIOLATION.STATUS.DATE": "",
"VIOLATION.DESCRIPTION": "WATCHMAN",
"VIOLATION.LOCATION": "",
"VIOLATION.INSPECTOR.COMMENTS": "",
"VIOLATION.ORDINANCE": "Maintain watchman from 4:00 PM to 8:00 AM for vacant and dangerous residential premises. (13-12-140)",
"INSPECTOR.ID": "BL00952",
"INSPECTION.NUMBER": 11233085,
"INSPECTION.STATUS": "CLOSED",
"INSPECTION.WAIVED": "N",
"INSPECTION.CATEGORY": "COMPLAINT",
"DEPARTMENT.BUREAU": "DEMOLITION",
"ADDRESS": "3836 W JACKSON BLVD",
"PROPERTY.GROUP": 248980,
"FULL_ADDR": "3836 W JACKSON BLVD",
"HOUSE_HIGH": 3836,
"SUFDIR": " ",
"XCOORD": 1150686.3596,
"YCOORD": 1898448.7671,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7222,
"latitude": 41.8772,
"bldg_id": "361919",
"feature_date": "2014-04-14"
},
{
"HOUSE_LOW": 3836,
"PRE": "W",
"STREET_NAME": "JACKSON",
"STREET_TYPE": "BLVD",
"ID": 4709081,
"VIOLATION.LAST.MODIFIED.DATE": "04/14/2014",
"VIOLATION.DATE": "2014/04/14",
"VIOLATION.CODE": "CN193110",
"VIOLATION.STATUS": "OPEN",
"VIOLATION.STATUS.DATE": "",
"VIOLATION.DESCRIPTION": "VACANT BUILDING - REGISTER",
"VIOLATION.LOCATION": "",
"VIOLATION.INSPECTOR.COMMENTS": "",
"VIOLATION.ORDINANCE": "Register vacant building within 30 days of it becoming vacant, or within 30 days after assuming ownership of an existing vacant building. (13-12-125(a)). Building must be kept in compliance with all vacant building requirements pursuant to 13-12-135. See Vacant Building Ordinance and registration form at https://ipiweb.cityofchicago.org/VBR",
"INSPECTOR.ID": "BL00952",
"INSPECTION.NUMBER": 11233085,
"INSPECTION.STATUS": "CLOSED",
"INSPECTION.WAIVED": "N",
"INSPECTION.CATEGORY": "COMPLAINT",
"DEPARTMENT.BUREAU": "DEMOLITION",
"ADDRESS": "3836 W JACKSON BLVD",
"PROPERTY.GROUP": 248980,
"FULL_ADDR": "3836 W JACKSON BLVD",
"HOUSE_HIGH": 3836,
"SUFDIR": " ",
"XCOORD": 1150686.3596,
"YCOORD": 1898448.7671,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7222,
"latitude": 41.8772,
"bldg_id": "361919",
"feature_date": "2014-04-14"
},
{
"HOUSE_LOW": 3836,
"PRE": "W",
"STREET_NAME": "JACKSON",
"STREET_TYPE": "BLVD",
"ID": 4709082,
"VIOLATION.LAST.MODIFIED.DATE": "04/14/2014",
"VIOLATION.DATE": "2014/04/14",
"VIOLATION.CODE": "CN196039",
"VIOLATION.STATUS": "OPEN",
"VIOLATION.STATUS.DATE": "",
"VIOLATION.DESCRIPTION": "POST OWNER INFO ON ABDN BLDG",
"VIOLATION.LOCATION": "",
"VIOLATION.INSPECTOR.COMMENTS": "",
"VIOLATION.ORDINANCE": "Post, on abandoned building, name, address and telephone of owner and owner's agent for managing, controlling or collecting rents on building so legible from nearest public street or sidewalk. (13-12-125)",
"INSPECTOR.ID": "BL00952",
"INSPECTION.NUMBER": 11233085,
"INSPECTION.STATUS": "CLOSED",
"INSPECTION.WAIVED": "N",
"INSPECTION.CATEGORY": "COMPLAINT",
"DEPARTMENT.BUREAU": "DEMOLITION",
"ADDRESS": "3836 W JACKSON BLVD",
"PROPERTY.GROUP": 248980,
"FULL_ADDR": "3836 W JACKSON BLVD",
"HOUSE_HIGH": 3836,
"SUFDIR": " ",
"XCOORD": 1150686.3596,
"YCOORD": 1898448.7671,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7222,
"latitude": 41.8772,
"bldg_id": "361919",
"feature_date": "2014-04-14"
},
{
"HOUSE_LOW": 3836,
"PRE": "W",
"STREET_NAME": "JACKSON",
"STREET_TYPE": "BLVD",
"ID": 4748724,
"VIOLATION.LAST.MODIFIED.DATE": "05/17/2016",
"VIOLATION.DATE": "2014/05/28",
"VIOLATION.CODE": "CN193039",
"VIOLATION.STATUS": "COMPLIED",
"VIOLATION.STATUS.DATE": "05/28/2014",
"VIOLATION.DESCRIPTION": "POST OWNER NAME OF VACNT BLDG",
"VIOLATION.LOCATION": "",
"VIOLATION.INSPECTOR.COMMENTS": "VIOLATION CLOSED PER PCR 1180",
"VIOLATION.ORDINANCE": "Post conspicuously name, address, and telephone of owner and manager of vacant residential premises. (13-12-125).",
"INSPECTOR.ID": "BL00900",
"INSPECTION.NUMBER": 11236585,
"INSPECTION.STATUS": "CLOSED",
"INSPECTION.WAIVED": "N",
"INSPECTION.CATEGORY": "COMPLAINT",
"DEPARTMENT.BUREAU": "DEMOLITION",
"ADDRESS": "3836 W JACKSON BLVD",
"PROPERTY.GROUP": 248980,
"FULL_ADDR": "3836 W JACKSON BLVD",
"HOUSE_HIGH": 3836,
"SUFDIR": " ",
"XCOORD": 1150686.3596,
"YCOORD": 1898448.7671,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7222,
"latitude": 41.8772,
"bldg_id": "361919",
"feature_date": "2014-05-28"
}
],
"demoPIN": []
}
],
"VacancyReports": [
{
"vacant311Bldg": [
{
"ADDRESS.STREET.NUMBER": "3836",
"ADDRESS.STREET.DIRECTION": "W",
"ADDRESS.STREET.NAME": "JACKSON",
"ADDRESS.STREET.SUFFIX": "BLVD",
"SERVICE.REQUEST.TYPE": "Vacant/Abandoned Building",
"SERVICE.REQUEST.NUMBER": "14-01143689",
"DATE.SERVICE.REQUEST.WAS.RECEIVED": "2014/07/17",
"LOCATION.OF.BUILDING.ON.THE.LOT..IF.GARAGE..CHANGE.TYPE.CODE.TO.BGD..": "Rear",
"IS.THE.BUILDING.DANGEROUS.OR.HAZARDOUS.": "",
"IS.BUILDING.OPEN.OR.BOARDED.": "Open",
"IF.THE.BUILDING.IS.OPEN..WHERE.IS.THE.ENTRY.POINT.": "",
"IS.THE.BUILDING.CURRENTLY.VACANT.OR.OCCUPIED.": "Vacant",
"IS.THE.BUILDING.VACANT.DUE.TO.FIRE.": "false",
"ANY.PEOPLE.USING.PROPERTY...HOMELESS..CHILDEN..GANGS.": "true",
"ZIP.CODE": 60624,
"Ward": 28,
"Police.District": 11,
"Community.Area": 26,
"FULL_ADDR": "3836 W JACKSON BLVD",
"HOUSE_LOW": 3836,
"HOUSE_HIGH": 3836,
"PRE": "W",
"STREET_NAME": "JACKSON",
"STREET_TYPE": "BLVD",
"SUFDIR": " ",
"XCOORD": 1150686.3596,
"YCOORD": 1898448.7671,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7222,
"latitude": 41.8772,
"bldg_id": "361919",
"feature_date": "2014-07-17"
},
{
"ADDRESS.STREET.NUMBER": "3836",
"ADDRESS.STREET.DIRECTION": "W",
"ADDRESS.STREET.NAME": "JACKSON",
"ADDRESS.STREET.SUFFIX": "BLVD",
"SERVICE.REQUEST.TYPE": "Vacant/Abandoned Building",
"SERVICE.REQUEST.NUMBER": "14-01555836",
"DATE.SERVICE.REQUEST.WAS.RECEIVED": "2014/09/15",
"LOCATION.OF.BUILDING.ON.THE.LOT..IF.GARAGE..CHANGE.TYPE.CODE.TO.BGD..": "Garage",
"IS.THE.BUILDING.DANGEROUS.OR.HAZARDOUS.": "",
"IS.BUILDING.OPEN.OR.BOARDED.": "Open",
"IF.THE.BUILDING.IS.OPEN..WHERE.IS.THE.ENTRY.POINT.": "",
"IS.THE.BUILDING.CURRENTLY.VACANT.OR.OCCUPIED.": "Vacant",
"IS.THE.BUILDING.VACANT.DUE.TO.FIRE.": "true",
"ANY.PEOPLE.USING.PROPERTY...HOMELESS..CHILDEN..GANGS.": "false",
"ZIP.CODE": 60624,
"Ward": 28,
"Police.District": 11,
"Community.Area": 26,
"FULL_ADDR": "3836 W JACKSON BLVD",
"HOUSE_LOW": 3836,
"HOUSE_HIGH": 3836,
"PRE": "W",
"STREET_NAME": "JACKSON",
"STREET_TYPE": "BLVD",
"SUFDIR": " ",
"XCOORD": 1150686.3596,
"YCOORD": 1898448.7671,
"STATUS1": "VALID",
"STATUS2": "ACTUAL",
"longitude": -87.7222,
"latitude": 41.8772,
"bldg_id": "361919",
"feature_date": "2014-09-15"
}
],
"vacant311PIN": []
}
]
}
nicklucius commented
A new version of the Axis App using this new version of building.json can be found here:
https://cdn.rawgit.com/Chicago/buildings/61f2e4af/notebooks/axis-prototype.html
tomschenkjr commented
@nicklucius - in your opinion, do you think the building.json format is sensible or does it need changes? From the code, it looks like it was easy to plot values, but I couldn't see the work you did in axis-prototype.R
nicklucius commented
@tomschenkjr - Yep, I think the format is good. You're not seeing the code because I'm actually using the data.frame from before the conversion to json to power this app, but only because I've already tested the back-and-forth conversion ad nauseum. I'm using jsonlite to do fromJSON and toJSON, and it handles the nested data perfectly.
The hard part was creating the nested data.frame in the first place, prior to the conversion to json. There's a trick to it--you have to create a data.frame first, and then add the nested data. If you try to create a data.frame from scratch with nested data, it doesn't work. This tripped me up for a while, and I was creating all sorts of convoluted list structures that couldn't take advantage of a lot of data.frame functions.
# Doesn't work
> df <- data.frame(address = "123 Fake St",
pin = list(c("12345","67890")))
> df
address c..12345....67890..
1 123 Fake St 12345
2 123 Fake St 67890
# Works
> df2 <- data.frame(address = "123 Fake St")
> df2$pin <- list(c("12345","67890"))
> df2
address pin
1 123 Fake St 12345, 67890
One thing to consider, though, is that some, but not all, of the nested features will grow, such as the lists of violations and 311 calls. In MongoDB, I believe those could be references to other MongoDB documents instead of the actual lists. That would eliminate duplication, and would allow interesting work to be done in the future using the relationships between entities. I could imagine researching a building, following a reference to a record that includes a person, then following a reference to a person entity, when then links to other buildings, businesses, and so on.
I think the summary counts could be done straight in MongoDB too. Either way, I now have the code in place to create the above-formatted JSON record for each building in Chicago.
Finally, there are some complications with inserting polygons into the file to make it a geojson, but we can take that up in #25.
tomschenkjr commented
Ok, that all makes sense.
I’d like to demo this to the WindyGrid team, so we start thinking about how to incorporate it into the application. Do you think you’d be ready for that? Perhaps a presentation in about 2 weeks?
From: Nick Lucius [mailto:notifications@github.com]
Sent: Friday, September 01, 2017 9:43 AM
To: Chicago/buildings <buildings@noreply.github.com>
Cc: Schenk, Tom <Tom.Schenk@cityofchicago.org>; Mention <mention@noreply.github.com>
Subject: Re: [Chicago/buildings] Add "summary" section to schema (#24)
@tomschenkjr<https://github.com/tomschenkjr> - Yep, I think the format is good. You're not seeing the code because I'm actually using the data.frame from before the conversion to json to power this app, but only because I've already tested the back-and-forth conversion ad nauseum. I'm using jsonlite to do fromJSON and toJSON, and it handles the nested data perfectly.
The hard part was creating the nested data.frame in the first place, prior to the conversion to JSON. There's a trick<https://stat.ethz.ch/pipermail/r-help/2007-February/125569.html> to it--you have to create a data.frame first, and then add the nested data. If you try to create a data.frame from scratch with nested data, it doesn't work. This tripped me up for a while, and I was creating all sorts of convoluted list() structures that couldn't take advantage of a lot of data.frame functions.
# Doesn't work
df <- data.frame(address = "123 Fake St",
pin = list(c("12345","67890")))
df
address c..12345....67890..
1 123 Fake St 12345
2 123 Fake St 67890
# Works
df2 <- data.frame(address = "123 Fake St")
df2$pin <- list(c("12345","67890"))
df2
address pin
1 123 Fake St 12345, 67890
One thing to consider, though, is that some, but not all, of the nested features that grow, such as the lists of violations and 311 calls. In MongoDB, I believe those could be references to other MongoDB documents<https://docs.mongodb.com/manual/tutorial/model-referenced-one-to-many-relationships-between-documents/> instead of the actual lists. That would eliminate duplication, and would allow interesting work to be done in the future using the relationships between entities. I could imagine researching a building, following a reference to a record that includes a person, then following a reference to a person entity, when then links to other buildings, businesses, and so on.
I think the summary counts could be done straight in MongoDB too. Either way, I now have the code in place to create the above-formatted JSON record for each building in Chicago.
Finally, there are some complications with inserting polygons into the file to make it a geojson, but we can take that up in #25<#25>.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#24 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ABkC0ajb4o4w3fzsFllJ2Y3ZGgLrAZUnks5seBfcgaJpZM4O6reR>.
…________________________________
This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail (or the person responsible for delivering this document to the intended recipient), you are hereby notified that any dissemination, distribution, printing or copying of this e-mail, and any attachment thereto, is strictly prohibited. If you have received this e-mail in error, please respond to the individual sending the message, and permanently delete the original and any copy of any e-mail and printout thereof.
nicklucius commented
Absolutely, I'd be happy to demo it for the team.
HeyZoos commented
Hey @nicklucius, thanks for demoing for us. Is there a way to get this file in its GeoJSON form? https://github.com/Chicago/buildings/blob/master/notebooks/combinedGeoJSON.Rds
nicklucius commented
nicklucius commented