mapshaper command
Roostaei110 opened this issue · 4 comments
First of all, I would like to express my gratitude for using mapshaper. I am encountering an issue and kindly ask for your assistance. I possess a Tabriz_houses_inf.csv file containing latitude and longitude data, as well as a Postal Codes map in JSON format (codgashttabriz.json) with gasht_name assigned as the postal code. My objective is to append a gasht_name column to the Tabriz_houses_inf.csv file. The below commands are implemented in the web API and are functioning correctly:
$ -points x=longitude y=latitude
$ -proj wgs84
[proj] Source and destination CRS are the same
$ -clip codgashttabriz
$ -join codgashttabriz target='Tabriz_houses_inf' fields='gasht_name'
I have installed mapshaper on my Linux server and executed the same commands (as follows); however, it did not produce the desired output.
mapshaper Tabriz_houses_inf.csv -points y=latitude x=longitude
mapshaper codgashttabriz.json-proj wgs84
mapshaper Tabriz_houses_inf.csv -clip codgashttabriz.json
mapshaper -join codgashttabriz.json keys='[latitude,longitude],gasht_name' Tabriz_houses_inf.csv
When executing join commands, the issue arises. I have attempted the following commands:
First command:
mapshaper Tabriz_houses_inf.csv -join codgashttabriz.json target='Tabriz_houses_inf' fields='gasht_name'
Error: [join] Unable to join polygon geometry to null geometry
Second command:
mapshaper Tabriz_houses_inf.csv -join codgashttabriz.json keys='[latitude,longitude],gasht_name'
Error: [join] Expected two key fields: a target field and a source field
The primary problem is with the line 'keys='[latitude,longitude],gasht_name'. The latitude and longitude values exist in the CSV file, while gasht_name is in the JSON file. Why does this solution work in the web API but not with the mapshaper command? How can I resolve this issue?
Hi!
The syntax of the web console and the command line are slightly different. You can run "history" in the web console to translate your console commands to their command line equivalents, but you'll probably have to change the paths of the input files, because the web console doesn't know where on your computer the input files came from.
I can see that keys='[latitude,longitude],gasht_name' in your example is wrong. The keys option is used when joining two layers by matching data values in so-called "key" fields. I think you want to do a different kind of join, called a spatial join, which doesn't use the keys option.
I think the solution will look more like this:
mapshaper -i Tabriz_houses_inf.csv \
-points \
-join codgashttabriz.json fields=gasht_name \
-o output.geojson
Good luck!
Hi @mbloch, thank you for your attention. Here are the steps I followed:
Step One: Running history in the web console:
mapshaper -i codgashttabriz.shp
-i Tabriz_houses_inf.csv
-points x=longitude y=latitude
-target codgashttabriz
-proj wgs84
-clip codgashttabriz
-join codgashttabriz target='Tabriz_houses_inf' fields='gasht_name'
-o target= format=dsv
Step Two: Writing in a python file:
mapshaper_commands = [
["mapshaper", "-i", shp_file],
["mapshaper", "-i", csv_file],
["mapshaper", csv_file, "-points", "x=longitude", "y=latitude"],
["mapshaper", shp_file, "-target", shp_file[:-4]],
["mapshaper", shp_file, "-proj", "wgs84"],
["mapshaper", csv_file, "-clip", shp_file[:-4]],
["mapshaper", '-join', shp_file[:-4], f'target={csv_file[:-4]}', 'fields=gasht_name'],
["mapshaper", csv_file, "-target", csv_file[:-4], '-o', 'target= format=dsv']
]
for line in mapshaper_commands:
subprocess.run(line)
The output is:
[i] Auto-detected number fields: code, unit_price, unit_price_En, elevator, parking, warehouse, meterage, latitude, longitude
[i] Auto-detected number fields: code, unit_price, unit_price_En, elevator, parking, warehouse, meterage, latitude, longitude
[points] 2,402 of 3,586 points are null
[wkt] unhandled parameter: Auxiliary_Sphere_Type
[i] Auto-detected number fields: code, unit_price, unit_price_En, elevator, parking, warehouse, meterage, latitude, longitude
Error: [clip] File not found (codgashttabriz)
Run mapshaper -h to view help
Error: [join] Missing layer: Tabriz_houses_inf
Run mapshaper -h to view help
[i] Auto-detected number fields: code, unit_price, unit_price_En, elevator, parking, warehouse, meterage, latitude, longitude
Error: [o] Missing layer: format=dsv
Run mapshaper -h to view help
Step Three: I changed line ["mapshaper", csv_file, "-clip", shp_file[:-4]] to ["mapshaper", csv_file, "-clip", shp_file] , which resolved the related error. The output is as:
[i] Auto-detected number fields: code, unit_price, unit_price_En, elevator, parking, warehouse, meterage, latitude, longitude
[i] Auto-detected number fields: code, unit_price, unit_price_En, elevator, parking, warehouse, meterage, latitude, longitude
[points] 2,402 of 3,586 points are null
[wkt] unhandled parameter: Auxiliary_Sphere_Type
[i] Auto-detected number fields: code, unit_price, unit_price_En, elevator, parking, warehouse, meterage, latitude, longitude
[wkt] unhandled parameter: Auxiliary_Sphere_Type
Error: [join] Missing layer: Tabriz_houses_inf
Run mapshaper -h to view help
[i] Auto-detected number fields: code, unit_price, unit_price_En, elevator, parking, warehouse, meterage, latitude, longitude
Error: [o] Missing layer: format=dsv
Run mapshaper -h to view help
Step Four: I encountered issues with the join command not recognizing Tabriz_houses_inf or Tabriz_houses_inf.csv. I tried changing the line related to join as ["mapshaper", '-join', shp_file[:-4], f'target={csv_file}', 'fields=gasht_name'], but I received the following error:
Error: [join] Missing layer: Tabriz_houses_inf.csv
Run mapshaper -h to view help
[i] Auto-detected number fields: code, unit_price, unit_price_En, elevator, parking, warehouse, meterage, latitude, longitude
Error: [o] Missing layer: format=dsv
I'm unsure about the last two lines and how to resolve them. I apologize for the lengthy text. I wanted to provide all the necessary details.
I suggest first getting your sequence of commands to work in an interactive shell, like bash. I gave you a set of shell commands to try in my earlier comment. Once you have something like that working, then you can port it to Python.
Your process has multiple steps -- I often start simple, with one editing command followed by a -o or -info command to check that the result is what I want. Then I add more commands until I have an entire pipeline of commands.
The first problem I see with your Python script is that you are splitting up your work into multiple subprocesses. The entire sequence of mapshaper commands should be contained in a single Python subprocess. Otherwise the results from one command will not be passed on to the next command. Files are not modified unless they are overwritten using the -o command.
You were right @mbloch, I should have written everything in one command. My problem is solved. I really enjoyed mapshaper.