Automated Detection of Street-Level Product Displays

2018 Urban Science Intensive Capstone
Sponsor: NYU mHealth
CUSP Mentors: Gregory Dobler, Federica Bianco, Thomas Kirchner

About the USI Program

NYU CUSP’s Urban Science Intensive (USI) Capstone program brings together student teams with government agencies or research partners to address real-world urban challenges through data. The USI Presentation event is the culmination of their four-month Capstone projects and marks the final presentation of the students’ work during their studies at CUSP.

Projects consist of team-based work on a pressing urban issue. Teams work with a project sponsor to define the problem, collect and analyze data, visualize the results, and, finally, formulate and deliver a possible solution. Student teams are challenged to utilize urban informatics within the context of city operations and planning, while considering political, social, and economic realities and data management and ethics. The goal of each project is to create impactful, replicable, and actionable results that inform data-driven urban operations and a new understanding of city dynamics.

Project Description

The project will leverage image processing by building and training deep convolutional neural networks (CNN), to characterize tobacco-related health risk across New York City (NYC). Point-of-sale tobacco (POST) products and marketing has a more immediate and comprehensive effect on consumer behavior than any other source of media and has been estimated to increase tobacco sales in the U.S. by 12%-28%. A considerable body of evidence indicates that tobacco-related disparities are maintained, in part, by POST marketing practices, and it has been shown that the tobacco industry uses advertising to target minority and lower income populations. The inability to empirically document a direct link between product exposure and behavior is among the primary reasons proposed point-ofsale regulations have often failed to withstand industry legal challenge in the past. The goal of this project is to map POST marketing practices longitudinally across NYC using automated detection of tobacco signage in Google Street View imaging data. The team will improve the current training set of tobacco signs, train a CNN for detection of that signage and recognition of tobacco brands, build a spatio-temporal database of POST marketing strategies, and relate the POST census to socioeconomic characteristics at the neighborhood level to search for evidence of disparities, compensatory tactics, new products, or pricing/promotional details.

Project Schedule

Gantt chart