A re-implementation of Kinect2Scratch for the online version of Snap/BYOB/Scratch.
kinect-2-snap
comes with two separate pieces -- a server that you need to
run on your local machine, and a XML file that you need to import into the online
version of BYOB. You will need to set up both in order to use this project.
Note: this project is currently still under active development and may still contain bugs or missing features.
- You need either a Windows 7 or Windows 8 PC (preferably a fast one).
- Download the correct drivers: a. If you own "Kinect for Windows", download and install the MS Kinect Runtime v1.8 b. If you own "Kinect for Xbox", download and install the MS Kinect SDK v1.8
- Setup your Kinect a. You must own a powered Kinect -- one with its own electricity plug. b. Plug the kinect c. Plug the USB lead on the Kinect into your PC. If you have other USB devices plugged in, you may need to remove them.
- Download and extract the latest release from https://github.com/Michael0x2a/kinect-2-snap/releases
- Navigate to the
kinect_server
folder. - Double-click on "kinect_server.exe"
- You can verify that the server is running by navigating to
http://localhost:5000/demo
in your web browser, and standing in front of the Kinect.
- Open the online Snap editor (located at http://snap.berkeley.edu/)
- Click the icon on the upper-left of the page, then click "import".
- Navigate to the
block_definitions' folder and select
kinect.xml` to import it. - The commands you need will now be placed under the "Sensing" category.
To run the server directly from the source code, you'll need to:
-
Clone this repo:
git clone https://github.com/Michael0x2a/kinect-2-snap.git
-
Install
flask
,flask-cors
, andpykinect
. If you havepip
installed, you can simply run the following commands from the command line:pip install flask pip install flask-cors pip install pykinect
-
Navigate into the
kinect_server
folder and runpython server.py
. You can then accesslocalhost:5000
to retrieve data.
If the Kinect server fails for any reason, here are some things you can try.
-
Make sure the Kinect is set up correctly.
The Kinect should be plugged into the power outlet and be plugged into your computer via USB. A green light should be shining on the Kinect.
-
Make sure you've installed the correct SDKs
If you own Kinect for Windows, you must install "MS Kinect Runtime v1.8". If you own Kinect for Windows, you must install "MS Kinect SDK v1.8". If you're not certain which one you own, install both.
Do not install the 2.0 SDK.
-
Unplug any other USB devices
Unplug anything hooked up to your computer via USB, except for the Kinect, and try again.
-
Wait a few seconds, and retry
If you've just plugged the Kinect in, then it'll take a few seconds for it to be ready. If the server fails for no apparent reason, then try again one or two times.
-
Unplug the Kinect, and reconnect it
Sometimes, unplugging the Kinect from your computer and plugging it back in will resolve any bizarre issues.
-
Email me
The Kinect server is accessible at localhost:5000
and provides the following
endpoints. Currently, if you call the endpoints as-is, the data will be returned
in a format that's designed to be easy for BYOB/Snap to parse. More information
about this data format will follow below.
If you add the query parameter ?format=json
to the end of the string, then the
data will be returned in JSON format.
-
localhost:5000
Returns all skeletal data from the Kinect.
-
localhost:5000/demo
Displays skeletal data on the screen. Requires Javascript.
-
localhost:5000/num_tracked
Returns the number of skeletons currently being tracked. Typically ranges from 0 to 2.
-
localhost:5000/skeletons
Returns data for all skeletons being tracked.
-
localhost:5000/skeletons/<num>
Returns data for that particular skeleton. Valid values are
1
or2
.If you provide
0
as the skeleton number, then the server will return any currently active skeleton. By default, the server will try and return skeleton 1, then try returning skeleton 2 if skeleton 1 is inactive. This way, you're guaranteed to get a skeleton so long as at least one person is in front of the kinect. -
localhost:5000/skeletons/<num>/<joint>
Returns data for that particular joint for that particular skeleton. Valid values are:
- FootLeft
- FootRight
- AnkleLeft
- AnkleRight
- KneeLeft
- KneeRight
- HipCenter
- HipLeft
- HipRight
- Spine
- HandLeft
- HandRight
- WristLeft
- WristRight
- ElbowLeft
- ElbowRight
- ShoulderCenter
- ShoulderLeft
- ShoulderRight
- Head
The match is case insensitive, and will ignore any dashes and underscores. For example, the following queries will behave identically:
http://localhost:5000/skeletons/1/AnkleLeft http://localhost:5000/skeletons/1/ankleleft http://localhost:5000/skeletons/1/ankle_left http://localhost:5000/skeletons/1/a-n-k-l-e_l-e-f-t
-
localhost:5000/skeletons/<num>/<joint>/<coord>
Returns the value of that particular axis on the joint. Valid values are X, Y, Z, and W. The X coordinate will range from about -240 to 240, and the Y coordinate will range from about -180 to 180. The Z coordinate is the distance of the joint from the Kinect camera in millimeters. The W coordinate is returned unchanged from the Kinect SDK.
When calling an endpoint to get skeletal data, the data will be returned with the x, y, z, and w coordinates of each joint separated by a space, and each joint separated by a newline. The joints will always be provided in the following format:
- FootLeft
- FootRight
- AnkleLeft
- AnkleRight
- KneeLeft
- KneeRight
- HipCenter
- HipLeft
- HipRight
- Spine
- HandLeft
- HandRight
- WristLeft
- WristRight
- ElbowLeft
- ElbowRight
- ShoulderCenter
- ShoulderLeft
- ShoulderRight
- Head