Attempting to use the config-sample.json
Opened this issue · 10 comments
Hello, I am getting an error, please see below
create config.json using the config-sample.json
cp config-sample.json config.json
Install gtfs-to-chart globally
npm install gtfs-to-chart -g
contents of config-sample.json
{
"agencies": [
{
"agency_key": "bart",
"url": "http://www.bart.gov/dev/schedules/google_transit.zip"
}
],
"sqlitePath": "/tmp/gtfs"
}
Running
gtfs-to-chart --configPath config.json
Output
Starting GTFS import for 1 file using SQLite database at /tmp/gtfs
Downloading GTFS from http://www.bart.gov/dev/schedules/google_transit.zip
Download successful
Importing GTFS from /tmp/tmp-141173-eank3fXmavG0/gtfs.zip
Importing - agency.txt - 1 lines imported
Importing - areas.txt - No file found
Importing - attributions.txt - No file found
Importing - calendar_dates.txt - 32 lines imported
Importing - calendar.txt - 13 lines imported
Importing - fare_attributes.txt - 2500 lines imported
Importing - fare_leg_rules.txt - No file found
Importing - fare_products.txt - No file found
Importing - fare_rules.txt - 2500 lines imported
Importing - fare_transfer_rules.txt - No file found
Importing - feed_info.txt - 1 lines imported
Importing - frequencies.txt - No file found
Importing - levels.txt - No file found
Importing - pathways.txt - No file found
Importing - routes.txt - 14 lines imported
Importing - shapes.txt - 38044 lines imported
Importing - stop_areas.txt - No file found
Importing - stop_times.txt - 61853 lines imported
Importing - stops.txt - 186 lines imported
Importing - transfers.txt - 41 lines imported
Importing - translations.txt - No file found
Importing - trips.txt - 4225 lines imported
Importing - calendar_attributes.txt - 11 lines imported
Importing - directions.txt - 14 lines imported
Importing - route_attributes.txt
Warning: Check route_attributes.txt for invalid data between lines 0 and 14.
file:///home/tmc/.nvm/versions/node/v16.19.1/lib/node_modules/gtfs-to-chart/node_modules/gtfs/lib/import.js:473
throw error;
^
[ SqliteError: route_attributes.route_id
- import.js:466 importLines
[lib]/[gtfs-to-chart]/[gtfs]/lib/import.js:466:7
- import.js:547 Parser.<anonymous>
[lib]/[gtfs-to-chart]/[gtfs]/lib/import.js:547:11
- node:events:513 Parser.emit
node:events:513:28
- readable:1358 endReadableNT
node:internal/streams/readable:1358:12
- task_queues:83 processTicksAndRejections
node:internal/process/task_queues:83:21
] {
code: 'SQLITE_CONSTRAINT_PRIMARYKEY'
}
Contents of route_attributes.txt
route_id,category,subcategory,running_way
1,2,201,1
2,2,201,1
3,2,201,1
4,2,201,1
5,2,201,1
6,2,201,1
7,2,201,1
8,2,201,1
11,2,201,1
12,2,201,1
19,3,301,1
20,3,301,1
2,2,201,1
1,2,201,1
Also, how would you recommend I get the data for Boston's MBTA?
I'm a little confused about sourcing that data
I see these data files https://transitfeeds.com/search?q=mbta
Thanks very much
Perhaps this is for Boston https://transitfeeds.com/p/mbta/64/latest
Not sure how to get MBTA data, this is my attempt:
Boston MBTA data
https://transitfeeds.com/p/mbta/64/latest
https://transitfeeds.com/p/mbta/64/latest/download
create config-mbta.json
{
"agencies": [
{
"agency_key": "mbta",
"url": "https://transitfeeds.com/p/mbta/64/latest/download/gtfs.zip"
}
],
"sqlitePath": "/tmp/gtfs"
}
Running
gtfs-to-chart --configPath config-mbta.json
Output
gtfs-to-chart --configPath config-mbta.json
Starting GTFS import for 1 file using SQLite database at /tmp/gtfs
Downloading GTFS from https://transitfeeds.com/p/mbta/64/latest/download/gtfs.zip
Error: Couldn’t download files
file:///home/tmc/.nvm/versions/node/v16.19.1/lib/node_modules/gtfs-to-chart/node_modules/gtfs/lib/import.js:41
throw new Error('Couldn’t download files');
^
[ Error: Couldn’t download files
- import.js:41 downloadFiles
[lib]/[gtfs-to-chart]/[gtfs]/lib/import.js:41:11
- task_queues:96 processTicksAndRejections
node:internal/process/task_queues:96:5
- import.js:597
[lib]/[gtfs-to-chart]/[gtfs]/lib/import.js:597:9
- index.js:7 async promiseMapSeries
[lib]/[gtfs-to-chart]/[promise-map-series]/index.js:7:18
- import.js:577 async importGtfs
[lib]/[gtfs-to-chart]/[gtfs]/lib/import.js:577:5
- gtfs-to-chart.js:43
[lib]/[gtfs-to-chart]/lib/gtfs-to-chart.js:43:7
- async Promise.all
- gtfs-to-chart.js:66
[lib]/[gtfs-to-chart]/bin/gtfs-to-chart.js:66:5
]
It looks like you can get MBTA's GTFS here: https://cdn.mbta.com/MBTA_GTFS.zip - try that URL
Also, try the gtfs-to-chart version 2.0.6 which was just released - this solves some issues with unzipping files in node.js v18 https://github.com/BlinkTagInc/gtfs-to-chart/releases/tag/2.0.6
Hmmm, still the same error, although a different line number is referenced
was
file:///home/tmc/.nvm/versions/node/v16.19.1/lib/node_modules/gtfs-to-chart/node_modules/gtfs/lib/import.js:473
now is
file:///home/tmc/.nvm/versions/node/v16.19.1/lib/node_modules/gtfs-to-chart/node_modules/gtfs/lib/import.js:482
Install latest version
npm install gtfs-to-chart -g
Output
added 1 package, removed 22 packages, and changed 318 packages in 15s
49 packages are looking for funding
run `npm fund` for details
Check version
gtfs-to-chart --version
Output
2.0.6
Run
gtfs-to-chart --configPath config.json
Output
Starting GTFS import for 1 file using SQLite database at /tmp/gtfs
Downloading GTFS from http://www.bart.gov/dev/schedules/google_transit.zip
Download successful
Importing GTFS from /tmp/tmp-178870-serzBY5d4W2p/gtfs.zip
Importing - agency.txt - 1 lines imported
Importing - areas.txt - No file found
Importing - attributions.txt - No file found
Importing - calendar_dates.txt - 32 lines imported
Importing - calendar.txt - 13 lines imported
Importing - fare_attributes.txt - 2500 lines imported
Importing - fare_leg_rules.txt - No file found
Importing - fare_products.txt - No file found
Importing - fare_rules.txt - 2500 lines imported
Importing - fare_transfer_rules.txt - No file found
Importing - feed_info.txt - 1 lines imported
Importing - frequencies.txt - No file found
Importing - levels.txt - No file found
Importing - pathways.txt - No file found
Importing - routes.txt - 14 lines imported
Importing - shapes.txt - 38044 lines imported
Importing - stop_areas.txt - No file found
Importing - stop_times.txt - 61853 lines imported
Importing - stops.txt - 186 lines imported
Importing - transfers.txt - 41 lines imported
Importing - translations.txt - No file found
Importing - trips.txt - 4225 lines imported
Importing - calendar_attributes.txt - 11 lines imported
Importing - directions.txt - 14 lines imported
Importing - route_attributes.txt
Warning: Check route_attributes.txt for invalid data between lines 0 and 14.
file:///home/tmc/.nvm/versions/node/v16.19.1/lib/node_modules/gtfs-to-chart/node_modules/gtfs/lib/import.js:482
throw error;
^
[ SqliteError: route_attributes.route_id
- import.js:475 importLines
[lib]/[gtfs-to-chart]/[gtfs]/lib/import.js:475:7
- import.js:556 Parser.<anonymous>
[lib]/[gtfs-to-chart]/[gtfs]/lib/import.js:556:11
- node:events:513 Parser.emit
node:events:513:28
- readable:1358 endReadableNT
node:internal/streams/readable:1358:12
- task_queues:83 processTicksAndRejections
node:internal/process/task_queues:83:21
- task_queues:65 runNextTicks
node:internal/process/task_queues:65:3
- timers:437 processImmediate
node:internal/timers:437:9
] {
code: 'SQLITE_CONSTRAINT_PRIMARYKEY'
}
I'm just noticing your reference to node v18, I am using v16
I will try node v18
Looks like it is still occurring even with node v18
nvm use v18
Output
Now using node v18.15.0 (npm v9.5.0)
Run
gtfs-to-chart --configPath config.json
Output
gtfs-to-chart: command not found
Install
npm install gtfs-to-chart -g
Output
added 319 packages in 14s
49 packages are looking for funding
run `npm fund` for details
Run
gtfs-to-chart --configPath config.json
Output
Starting GTFS import for 1 file using SQLite database at /tmp/gtfs
Downloading GTFS from http://www.bart.gov/dev/schedules/google_transit.zip
Download successful
Importing GTFS from /tmp/tmp-181995-1WN38QCkMYFO/gtfs.zip
Importing - agency.txt - 1 lines imported
Importing - areas.txt - No file found
Importing - attributions.txt - No file found
Importing - calendar_dates.txt - 32 lines imported
Importing - calendar.txt - 13 lines imported
Importing - fare_attributes.txt - 2500 lines imported
Importing - fare_leg_rules.txt - No file found
Importing - fare_products.txt - No file found
Importing - fare_rules.txt - 2500 lines imported
Importing - fare_transfer_rules.txt - No file found
Importing - feed_info.txt - 1 lines imported
Importing - frequencies.txt - No file found
Importing - levels.txt - No file found
Importing - pathways.txt - No file found
Importing - routes.txt - 14 lines imported
Importing - shapes.txt - 38044 lines imported
Importing - stop_areas.txt - No file found
Importing - stop_times.txt - 61853 lines imported
Importing - stops.txt - 186 lines imported
Importing - transfers.txt - 41 lines imported
Importing - translations.txt - No file found
Importing - trips.txt - 4225 lines imported
Importing - calendar_attributes.txt - 11 lines imported
Importing - directions.txt - 14 lines imported
Importing - route_attributes.txt
Warning: Check route_attributes.txt for invalid data between lines 0 and 14.
file:///home/tmc/.nvm/versions/node/v18.15.0/lib/node_modules/gtfs-to-chart/node_modules/gtfs/lib/import.js:482
throw error;
^
[ SqliteError: route_attributes.route_id
- import.js:475 importLines
[lib]/[gtfs-to-chart]/[gtfs]/lib/import.js:475:7
- import.js:556 Parser.<anonymous>
[lib]/[gtfs-to-chart]/[gtfs]/lib/import.js:556:11
- node:events:513 Parser.emit
node:events:513:28
- readable:1359 endReadableNT
node:internal/streams/readable:1359:12
- task_queues:82 process.processTicksAndRejections
node:internal/process/task_queues:82:21
] {
code: 'SQLITE_CONSTRAINT_PRIMARYKEY'
}
Node.js v18.15.0
Great news - MBTA loads and makes charts - looks like the problem file route_attributes.txt does not exist in the MBTA file
big file with lots of rows
gtfs-to-chart --configPath config-mbta.json
Output
Starting GTFS import for 1 file using SQLite database at /tmp/gtfs
Downloading GTFS from https://cdn.mbta.com/MBTA_GTFS.zip
Download successful
Importing GTFS from /tmp/tmp-182845-5Sj01ZrXnN4T/gtfs.zip
Importing - agency.txt - 1 lines imported
Importing - areas.txt - 32 lines imported
Importing - attributions.txt - No file found
Importing - calendar_dates.txt - 54 lines imported
Importing - calendar.txt - 90 lines imported
Importing - fare_attributes.txt - No file found
Importing - fare_leg_rules.txt - 456 lines imported
Importing - fare_products.txt - 87 lines imported
Importing - fare_rules.txt - No file found
Importing - fare_transfer_rules.txt - 37 lines imported
Importing - feed_info.txt - 1 lines imported
Importing - frequencies.txt - No file found
Importing - levels.txt - 74 lines imported
Importing - pathways.txt - 7999 lines imported
Importing - routes.txt - 219 lines imported
Importing - shapes.txt - 258537 lines imported
Importing - stop_areas.txt - 758 lines imported
Importing - stop_times.txt - 1605036 lines imported
Importing - stops.txt - 9743 lines imported
Importing - transfers.txt - 5312 lines imported
Importing - translations.txt - No file found
Importing - trips.txt - 65139 lines imported
Importing - calendar_attributes.txt - 90 lines imported
Importing - directions.txt - 378 lines imported
Completed GTFS import for 1 agency
mbta: Generating charts [===================================----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 35/219
Warning: No trips found for route CR-Foxboro on May 5, 2023
mbta: Generating charts [============================================================================================================================-----------------------------------------------------------------------------------------------] 124/219
Warning: No trips found for route Shuttle-Generic-CommuterRail-South on May 5, 2023
Warning: No trips found for route Shuttle-Generic-CommuterRail-North on May 5, 2023
Warning: No trips found for route Shuttle-Generic-CommuterRail on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Blue on May 5, 2023
Warning: No trips found for route Shuttle-Generic on May 5, 2023
mbta: Generating charts [===============================================================================================================================================================================--------------------------------------------] 175/219
Warning: No trips found for route 195 on May 5, 2023
mbta: Generating charts [============================================================================================================================================================================================-------------------------------] 188/219
Warning: No trips found for route Shuttle-JFKPark on May 5, 2023
mbta: Generating charts [==============================================================================================================================================================================================-----------------------------] 190/219
Warning: No trips found for route Shuttle-JFKKendall on May 5, 2023
mbta: Generating charts [=================================================================================================================================================================================================--------------------------] 193/219
Warning: No trips found for route Shuttle-GovernmentCenterWonderland on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Worcester on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Red-Trunk on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Red-Braintree on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Red-Ashmont on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Red on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Providence on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Orange on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Newburyport on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Needham on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Middleborough on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Mattapan on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Lowell on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Kingston on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Haverhill on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Green-Trunk on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Green-E on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Green-D on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Green-C on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Greenbush on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Green-B on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Green on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Franklin on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Fitchburg on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Fairmount on May 5, 2023
Warning: No trips found for route Shuttle-Generic-Elevator on May 5, 2023
mbta: Generating charts [===========================================================================================================================================================================================================================] 219/219
mbta: charts created at /home/tmc/ap/vol6/gtfs-to-chart-stringlines/charts/mbta
I am wondering the best way to view the charts?
I can open the index file on my machine and browse each one, but I can't view them via GitHub at my repo
https://github.com/coding-to-music/gtfs-to-chart-stringlines/tree/main/charts/mbta
Is there an existing React or NextJS or Angular app that knows how to view the charts?
Also, when I look at a chart, it appears to only contain a single inbound and outbound row, perhaps from last night's last run of the day?
It does not show the trips throughout the day, such as this similar one I saw the other day on Twitter, not sure how it was generated https://twitter.com/ndhapple/status/1654169426944458773
I tried out https://github.com/brendannee/whereisbart.com very cool
And magically was able to deploy my clone to vercel, works, amazing! https://whereisbart-leaflet-google-transit.vercel.app
I tried the same with this, I was able to deploy via vercel --prod
and specify the file locations are in charts/mbta
pretty cool, it works!! https://gtfs-to-chart-stringlines.vercel.app
I wonder why only the last runs of the day are visible?
For example
-
Green-B https://gtfs-to-chart-stringlines.vercel.app/charts/B.html
-
Green-C https://gtfs-to-chart-stringlines.vercel.app/charts/C.html
-
Green-D https://gtfs-to-chart-stringlines.vercel.app/charts/D.html
-
Green-E https://gtfs-to-chart-stringlines.vercel.app/charts/E.html
-
Orange Line https://gtfs-to-chart-stringlines.vercel.app/charts/Orange%20Line.html
-
Blue https://gtfs-to-chart-stringlines.vercel.app/charts/Blue%20Line.html
-
Nothing for the Red Line https://gtfs-to-chart-stringlines.vercel.app/charts/Red%20Line.html