/whycon

WhyCon localization system - non-ROS version used in the experiments for the JINT paper

Primary LanguageC++MIT LicenseMIT

WhyCon

A precise, efficient and low-cost localization system

WhyCon is a version of a vision-based localization system that can be used with low-cost web cameras, and achieves millimiter precision with very high performance. The system is capable of efficient real-time detection and precise position estimation of several circular markers in a video stream. It can be used both off-line, as a source of ground-truth for robotics experiments, or on-line as a component of robotic systems that require real-time, precise position estimation. WhyCon is meant as an alternative to widely used and expensive localization systems. It is fully open-source. WhyCon-orig is WhyCon's original, minimalistic version that was supposed to be ROS and openCV independent.

WhyCon example application (video) Scenario description
WhyCon applications -precise docking to a charging station (EU project STRANDS),
-fitness evaluation for self-evolving robots (EU proj. SYMBRION),
-relative localization of UAV-UGV formations (CZ-USA project COLOS),
-energy source localization in (EU proj REPLICATOR),
-robotic swarm localization (EU proj HAZCEPT).

The WhyCon system was developed as a joint project between the University of Buenos Aires, Czech Technical University and University of Lincoln, UK. The main contributors were Matias Nitsche, Tom Krajnik and Jan Faigl. Each of these contributors maintains a slightly different version of WhyCon.

WhyCon version Application Main features Maintainer
WhyCon-orig general 2D, 3D, ROS, lightweight, autocalibration Tom Krajnik
WhyCon-ROS general 2D, ROS Matias Nitsche
SwarmCon μ-swarms 2D, individual IDs, autocalibration Tom Krajnik
Caspa-WhyCon UAVs embedded, open HW-SW solution Jan Faigl
Social-card HRI ROS, allows to command a robot Tom Krajnik

Where is it described ?

WhyCon was first presented on International Conference on Advanced Robotics 2013 [2], later in the Journal of Intelligent and Robotics Systems [1] and finally at the Workshop on Open Source Aerial Robotics during the International Conference on Intelligent Robotic Systems, 2015 [3]. Its early version was also presented at the International Conference of Robotics and Automation, 2013 [4]. An extension of the system, which used a necklace code to add ID's to the tags, achieved a best paper award at the SAC 2017 conference [5]. If you decide to use this software for your research, please cite WhyCon using the one of the references provided in this bibtex file.

Setting up WhyCon in ROS

Quick setup for initial testing

  1. Have ROS Kinetic and appropriate camera driver installed. Also have a calibrated camera with distortion model "plumb bob".
  2. Install the required libraries
sudo apt-get install libsdl1.2-dev libsdl-ttf2.0-dev libncurses5-dev graphicsmagick-libmagick-dev-compat
  1. Download the code from GitHub into a catkin workspace.
  2. Compile the code - just type catkin_make in workspace directory.
  3. Source setup script in package directory into shell enviroment e.g. source devel/setup.bash
  4. Download, resize and print one circular pattern - you have the pattern also in the id/test.pdf file.
  5. Run code by roslaunch and remap subsribed camera topics either on start up through arguments
roslaunch whycon_ros whycon.launch camInfo:=/<camera>/camera_info camRaw:=/<camera>/image_raw

or rewrite file whycon.launch so default values of tags arg called camInfo and camRaw will match topics camera_info and image_raw. Then it's just

roslaunch whycon_ros whycon.launch
  1. Unless modifying code, whycon is started with max 100 markers to look for. This value has to be eventually changed in the code because of start up allocation of memory.
  2. If using patterns with encoded ID keep the option identify turned on in rqt_reconfigure and if without ID, then turn it off!!!
  3. You should see the image with some numbers below the circle. Pressing D shows the segmentation result.
  4. You can change the parameters in rqt_reconfigure which should open together with whycon GUI.

Generating tags with ID

  1. Folder id/ contains code which generates tags with IDs.
  2. Move to the id folder and type make.
  3. Run ./whycon-id-gen followed by a number of bits and it will create tags in the working directory.
  4. Other program parameters are specified in help ./whycon-id-gen -h
  5. Number of ID bits has to be then passed to whycon on start up. The default value is 5.
roslaunch whycon_ros whycon.launch [...] idBits:=...
  1. Other ID parameters are treated the same way. ID samples as idSamples:=... and Hamming distance as hammingDist:=...

Setting up the coordinate system

  1. If you have resized the markers (their default size is 122mm), then adjust their diameter in the rqt_reconfigure.
  2. Print additional four circular markers and place to the corners of your (reclangular) operational space.
  3. Position and fixate your camera so that it has all four circles in it's field of view.
  4. Run whycon and modify the dimensions of the operation space in the rqt_reconfigure - the system will now assume that the four markers are at positions [0,0],[fieldLength,0], [0,fieldWidth],[fieldLength,fieldWidth].
  5. Adjust the parameter numBots - the number of patterns you want to track plus 4.
  6. Once all the patterns are found, press a and the four outermost patterns will be used to calculate the coordinate system.
  7. Alternatively, you can press r and then click the four circles that define the coordinate system.
  8. Pressing 1 should show you the patterns' positions in camera-centric coordinates (x-axis equals to camera optical axis), pressing 2 and 3 will display marker coordinates in user-defined 2D or 3D coordinate systems.
  9. Pressing +,- changes the number of localized patterns.

Logs, GUI, recording topics

  1. GUI can be omitted by passing argument useGui:=false at strat up.
  2. Video and communication topics can be save using rosbag.

GUI key binding

  • h - show/hide help

  • + - increase the number of tracked patterns

  • - - decrease the number of tracked patterns

  • 1 - camera coordinate system

  • 2 - planar coordinate system

  • 3 - 3D coordinate system

  • l - draw/hide coordinates

  • d - draw/hide segmentation outcome and print debug info

  • s - save current image

  • a - autocalibration (searches for 4 outermost patterns and uses them to establisht the coordinate system)

  • r - manual calibration (click the 4 calibration patterns with mouse)

  • ESC - exit WhyCon

  • Individual patterns to be tracked can be selected by a mouse click.

  • m and n are recommended to use only with one tracked pattern.

  • m - prints camera coords and eigenvalues of the first pattern

  • n - prints image coords and eigenvalues of the first pattern

Topics

Published
  1. /whycon_ros/markers - you can find its header files in msg folder
    • Header header
    • whycon_ros/Marker[] markers
      • float32 u # camera coordinate
      • float32 v # camera coordinate
      • int32 size # size of the segment in pixels
      • int8 id # ID of pattern
      • geometry_msgs/Pose position # position with quaternion as orientation
      • geometry_msgs/Vector3 rotation # vector of Euler angles - pitch, roll, yaw
Subscribed
  1. /<camera>/camera_info - camera matrix and distortion coeffs
  2. /<camera>/image_raw - raw image data without correction

Some additional remarks

  1. At this point, you can start experimenting with the syste by adding whatever features you might think useful.
  2. We have tried to comment the code so an experienced programmer should be able to alter the system accordingly. However, if you have any questions regarding the code, feel free to contact Tom Krajnik or Matias Nitsche
  3. If you use this localization system for your research, please don't forget to cite at least one relevant paper from these bibtex records.
  4. Have fun!

Dependencies

All the following libraries are probably in your packages.

  1. libsdl1.2-dev for graphical user interface.
  2. libsdl-ttf2.0-dev to print stuff in the GUI.
  3. libncurses5-dev to print stuff on the terminal.
  4. graphicsmagick-libmagick-dev-compat to generate IDs.

References

  1. T. Krajník, M. Nitsche et al.: A Practical Multirobot Localization System. Journal of Intelligent and Robotic Systems (JINT), 2014. [bibtex].
  2. T. Krajník, M. Nitsche et al.: External localization system for mobile robotics. International Conference on Advanced Robotics (ICAR), 2013. [bibtex].
  3. M. Nitsche, T. Krajník et al.: WhyCon: An Efficent, Marker-based Localization System. IROS Workshop on Open Source Aerial Robotics, 2015. [bibtex].
  4. J. Faigl, T. Krajník et al.: Low-cost embedded system for relative localization in robotic swarms. International Conference on Robotics and Automation (ICRA), 2013. [bibtex].
  5. P. Lightbody, T. Krajník et al.: A versatile high-performance visual fiducial marker detection system with scalable identity encoding.Symposium on Applied Computing, 2017.[bibtex].

Acknowledgements

The development of this work is currently supported by the Czech Science Foundation project 17-27006Y STRoLL. In the past, the work was supported by EU within its Seventh Framework Programme project ICT-600623 STRANDS. The Czech Republic and Argentina have given support through projects 7AMB12AR022, ARC/11/11 and 13-18316P. We sincerely acknowledge Jean Pierre Moreau for his excellent libraries for numerical analysis that we use in our project.