OpenUxAS is developed by the Air Force Research Laboratory, Aerospace System Directorate, Power and Control Division. The LMCP specification and all source code for OpenUxAS is publicaly released under the Air Force Open Source Agreement Version 1.0. See LICENSE.md for complete details. The Air Force Open Source Agreement closely follows the NASA Open Source Agreement Verion 1.3. NOTE the terms of the license include registering use of the software by emailing afrl.rq.opensource@us.af.mil.
UxAS consists of a collection of modular services that interact via a common message passing architecture. Similar in design to Robot Operating System (ROS), each service subscribes to messages in the system and responds to queries. UxAS uses the open-source library ZeroMQ to connect all services to each other. The content of each message conforms to the Light-weight Message Control Protocol (LMCP) format. Software classes providing LMCP message creation, access, and serialization/deserialization are automatically generated from simple XML description documents (see the LmcpGen project). These same XML descriptions detail the exact data fields, units, and default values for each message. Since all UxAS services communicate with LMCP formatted messages, a developer can quickly determine the input/output data for each service. In a very real sense, the message traffic in the system exposes the interaction of the services that are required to achieve autonomous behavior.
Consider a simple example: the automated construction of the flight pattern to conduct surveillance of geometric lines (e.g. perimeters, roads, coasts). A “line search task” message describes the line to be imaged and the desired camera angle. Using this input description, a line search service calculates the appropriate waypoints to achieve the proper view angle. When the UAV arrives at the first waypoint corresponding to the line search task, the line search service continuously updates the desired camera pointing location to smoothly step the camera along the intended route.
In addition to surveillance pattern automation, UxAS contains services that automate route planning, coordinate behavior among multiple vehicles, connect with external software, validate mission requests, log and diagram message traffic, and optimize task ordering. In all, UxAS has approximately 30 services.
A core functionality provided by UxAS is the mechanism to calculate near-optimal task allocation across teams of unmanned vehicles. With a collection of tasks that require servicing and a pool of vehicles available to service those tasks, UxAS is able to determine which vehicle should do which task in the proper order. This task assignment pipeline is carried out by a series of services working together in a complex sequence.
Try:
mkdir -p /home/$USER/UxAS_pulls
cd /home/$USER/UxAS_pulls
git clone https://github.com/afrl-rq/OpenUxAS.git
cd OpenUxAS
git checkout BRANCH
./install_most_deps.sh
./checkout_plus_config.sh -d /home/$USER/UxAS_pulls BRANCH
./build_documentation
replacing BRANCH with the branch of OpenUxAS that you want (e.g., develop, architecture, rta ...). (It's recommended that you use /home/$USER/UxAS_arch
for the directory if you're using the archtiecture
branch, rather than /home/$USER/UxAS_pulls
for the develop
branch.)
Make sure you follow the instructions in the terminal window, and press a key once you're ready to move to the next set of instructions.
(Note that if you see a '$' prompt during install_most_deps.sh, that you'll need to type 'Ctrl-C' and 'Ctrl-D' once each to continue running the script.)
To test OpenUxAS 'example 2', try:
-
In terminal 1:
cd /home/$USER/UxAS_pulls/OpenUxAS/examples/02_Example_WaterwaySearch
./runAMASE_WaterwaySearch.sh
-
Press the 'play' button in the AMASE simulation player.
-
In terminal 2:
cd /home/$USER/UxAS_pulls/OpenUxAS/examples/02_Example_WaterwaySearch
./runUxAS_WaterwaySearch.sh
If you need to recompile OpenUxAS later, try:
cd ~/UxAS_pulls/OpenUxAS
ninja -C build all
If you need to pull the newest versions of the UxAS code from the server and recompile, try:
cd /home/$USER/UxAS_pulls/OpenUxAS
./checkout_plus_config.sh -d /home/$USER/UxAS_pulls BRANCH
replacing BRANCH with the branch of OpenUxAS that you want (e.g., develop, architecture, rta ...).
Alternately, if you change your mind after the fact and want to use (e.g.) the architecture branch for things, try:
cd /home/$USER/UxAS_pulls/OpenUxAS
./checkout_plus_config.sh -d /home/$USER/UxAS_pulls architecture
The primary tools and dependencies to obtain, build, document, and simulate UxAS are:
- Git
- OpenGL
- UUID library
- Boost
- Python 3
- Meson
- Ninja
- LmcpGen
- OpenAMASE (optional, simulation)
- NetBeans with Java JDK (optional, simulation)
- Doxygen (optional, documentation)
- LaTeX (optional, documentation)
The UxAS build system will download and compile the following external libraries
- Google Test
- ZeroMQ (zeromq, czmq, cppzmq, zyre)
- Sqlite
- Zlib
- Minizip
- Serial
Libraries for XML and GPS message parsing have numerous forks without centralized repository control. Code to build the following libraries is included with UxAS
- PugiXML
- TinyGPS
For an Ubuntu 16.04 or Mac OS X system with the listed prerequisite tools installed, UxAS should build from source without issue. Support for Windows is available on Windows 7 and 10 using Visual Studio.
The following is a bash script that helps to partially-automate the "installing prerequisite tools" processes that are documented in this README.md file below.
This has been tested-working on Ubuntu 16.04, as of 2017-05-23.
- Download the script from the OpenUxAS repository (install_most_deps.sh) OR
cd
to yourgit clone
d OpenUxAS directory - Run the script at the terminal:
./install_most_deps.sh
- Follow the on-screen instructions
Note that the most up-to-date instructions on the dependencies-needed for UxAS are available below.
- Install
git
: in terminalsudo apt-get install git
sudo apt-get install gitk
- Install OpenGL development headers: in terminal
sudo apt-get install libglu1-mesa-dev
- Install unique ID creation library: in terminal
sudo apt-get install uuid-dev
- Install Boost libraries (optional but recommended; see external dependencies section): in terminal
sudo apt-get install libboost-filesystem-dev libboost-regex-dev libboost-system-dev
- Install doxygen and related packages (optional): in terminal
sudo apt-get install doxygen
sudo apt-get install graphviz
sudo apt-get install texlive
sudo apt-get install texlive-latex-extra
- Install pip3: in terminal
sudo apt install python3-pip
sudo -H pip3 install --upgrade pip
- Install ninja build system: in terminal
sudo -H pip3 install ninja
- Install meson build configuration: in terminal
sudo -H pip3 install meson
- Ensure dependency search for meson is supported: in terminal
sudo apt-get install pkg-config
- Install python plotting capabilities (optional): in terminal
sudo apt install python3-tk
sudo -H pip3 install matplotlib
sudo -H pip3 install pandas
- Install NetBeans and Oracle Java JDK (optional)
- Download the Linux x64 version
- Run downloaded install script: in terminal
cd ~/Downloads; sh jdk-8u131-nb-8_w-linux-x64.sh
- Click
Next
three times, thenInstall
- Enable C/C++ plug-in in NetBeans (optional)
- Open NetBeans (in Ubuntu search, type
NetBeans
) - Choose Tools->Plugins from the top menu
- In the
Available Plugins
tab, search forC++
- Select
C/C++
and clickInstall
- Open NetBeans (in Ubuntu search, type
- Install Oracle Java run-time (required for LmcpGen): in terminal
sudo add-apt-repository ppa:webupd8team/java
sudo apt update; sudo apt install oracle-java8-installer
sudo apt install oracle-java8-set-default
- Install XCode
- Enable commandline tools: in terminal
xcode-select --install
- Install
homebrew
(must be administrator): in terminalsudo ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
- Add
homebrew
to path: in terminalecho $(export PATH="/usr/local/bin:$PATH") >> ~/.bash_profile
- Install
git
: in terminalbrew install git
- Install unique ID library: in terminal
brew install ossp-uuid
- Install Boost library and configure it in a fresh shell: in terminal
brew install boost
echo 'export BOOST_ROOT=/usr/local' >> ~/.bash_profile
bash
- Install
doxygen
and related packages (optional): in terminalbrew install doxygen
brew install graphviz
brew cask install mactex
- Install pip3: in terminal
brew install python3
- Install ninja build system: in terminal
brew install cmake
brew install pkg-config
sudo -H pip3 install scikit-build
sudo -H pip3 install ninja
- Install meson build configuration: in terminal
sudo -H pip3 install meson
- Install python plotting capabilities (optional): in terminal
sudo -H pip3 install matplotlib
sudo -H pip3 install pandas
- Install Oracle Java run-time (required for LmcpGen)
- Install NetBeans and Oracle Java JDK (optional)
- Download the Mac OSX version
- Install .dmg
- Enable C/C++ plug-in in NetBeans (optional)
- Open NetBeans
- Choose Tools->Plugins from the top menu
- In the
Available Plugins
tab, search forC++
- Select
C/C++
and clickInstall
- Install Visual Studio 2017 Community Edition
- Ensure C++ selected in
Workloads
tab - Ensure
Git for Windows
is selected inIndividual components
tab
- Ensure C++ selected in
- Install Git with Bash shell
- Install Python 3
- Make sure to check
Add Python 3.6 to PATH
- Choose standard install (
Install Now
, requires admin) - Verify installation by:
python --version
incmd
prompt - Verify pip is also installed:
pip --version
incmd
prompt - If unable to get python on path, follow this answer using location
C:\Users\[user]\AppData\Local\Programs\Python\Python36-32\
- Make sure to check
- Install meson
- In
cmd
prompt with admin priviledges:pip install meson
- In
- Install Boost
- Note: the above link is for VS2017 pre-compiled libraries. To compile from source, you must install at the location:
C:\local\boost_1_64_0
- Note: the above link is for VS2017 pre-compiled libraries. To compile from source, you must install at the location:
- Pull UxAS repositories (from Git Bash shell)
git -c http.sslVerify=false clone https://github.com/afrl-rq/OpenUxAS.git
git -c http.sslVerify=false clone https://github.com/afrl-rq/LmcpGen.git
- Auto-create the UxAS messaging library
- Download released executable from GitHub
- Place
LmcpGen.jar
inLmcpGen/dist
folder - In the root UxAS directory, run
RunLmcpGen.bat
- Prepare build
- Open VS command prompt (Tools -> Visual Studio Command Prompt)
python prepare
meson.py build --backend=vs
- Build project with Visual Studio
- Open project file
OpenUxAS.sln
in theOpenUxAS/build
directory - In the Solution Explorer, right-click the
uxas
project, and clickBuild
- Open project file
- The Visual Studio backend for Meson mostly works, but will fail when regenerating build files. If you modify one of the
meson.build
files, delete thebuild
directory and runmeson.py build --backend=vs
again. - The UxAS test suite uses some hardcoded POSIX-style paths, and so does not currently work on Windows.
Configure UxAS and Related Projects + Building at the Command Line on Ubuntu Linux / Bash on Ubuntu on Windows -or- Mac OS X (Partially-Automated)
The following is a bash script that helps to partially-automate the "configure UxAS and related projects" and "building at the command line" processes that are documented in this README.md file below.
This has been tested-working on Ubuntu 16.04, as of 2017-05-23.
- Download these two scripts from the OpenUxAS repository OR
cd
to yourgit clone
d OpenUxAS directorycheckout_plus_config.sh
get_dlvsco_wd_f.sh
- Run the
checkout_plus_config.sh
script at the terminal:- If you want to download the .jar files for OpenAMASE and LmcpGen, try:
./checkout_plus_config.sh -d
- If you want to compile the .jar files for OpenAMASE and LmcpGen, try:
./checkout_plus_config.sh -c
- If you want to download the .jar files for OpenAMASE and LmcpGen, try:
- Follow the on-screen instructions
Note that this sets up your UxAS workspace under a default directory (/home/$USER/UxAS_pulls
). If you want to specify a workspace other than the default, then pass the absolute path to the script as a second argument when calling the script (e.g., ./checkout_plus_config.sh -d /home/$USER/my_checkout_dir
).
Expected file system layout:
./
OpenAMASE
/OpenAMASE
/config
/data
/dist
OpenAMASE.jar <-- add this here to avoid compilation
/docs
/example scenarios
/lib
/native
/nbproject
/run
/src
LmcpGen
/dist
LmcpGen.jar <-- add this here to avoid compilation
/nbproject
/src
OpenUxAS
/3rd
/doc
/examples
/mdms
/resources
/src
/tests
/wrap_patches
-
EITHER Checkout + compile OpenAMASE (optional)
- File system layout: OpenAMASE should be a sibling to OpenUxAS (see above)
- Checkout:
git clone https://github.com/afrl-rq/OpenAMASE.git
- Compile: Load provided Netbeans project, click
Build
OR Download OpenAMASE (optional)
- File system layout: OpenAMASE should be a sibling to OpenUxAS (see above)
- Download: from GitHub
- Place
OpenAMASE.jar
inOpenAMASE/OpenAMASE/dist
folder
-
EITHER Checkout + compile LmcpGen
- File system layout: LmcpGen should be a sibling to OpenUxAS (see above)
- Checkout:
git clone https://github.com/afrl-rq/LmcpGen.git
- Compile: Load provided Netbeans project, click
Build
OR Download LmcpGen
- File system layout: LmcpGen should be a sibling to OpenUxAS (see above)
- Download: from GitHub
- Place
LmcpGen.jar
inLmcpGen/dist
folder
-
Auto-generate source code for LMCP libraries: in terminal in
OpenUxAS
directory- Assuming that in the file system, LmcpGen is at the same level as
OpenUxAS
(see above) sh RunLmcpGen.sh
- Assuming that in the file system, LmcpGen is at the same level as
-
Prepare UxAS specific patches to external libraries: in terminal in
OpenUxAS
directory./prepare
The above preparation (i.e. ./prepare
) needs to be done prior to the first build and any
time a file is modified in one of the /3rd/wrap_patches
subdirectories or the /3rd/*.wrap.tmpl
files.
This also needs to be done any time you move or rename your source tree.
- Configure for release build: in terminal
meson build --buildtype=release
- Configure for debug build: in terminal
meson build_debug --buildtype=debug
- These two steps only need to be done prior to the first build. If you modify the Meson files, just build as normal in step 3 and the changes will be automatically incorporated.
- Build UxAS: in terminal
ninja -C build all
- This step is the only step necessary in day-to-day development work. It's
the Meson equivalent of
make all
. Note that the name ofninja
may differ by distro. On Fedora, for example, it'sninja-build
. - To clean the build, add the
clean
target at the end of your ninja command:ninja -C build clean
- Run UxAS tests: in terminal
ninja -C build test
- Confirm all tests passed
- Open NetBeans
- Select File->New Project
- Choose
C/C++ Project with Existing Sources
and clickNext
- Specify the
OpenUxAS
folder - Select the
Custom
option underSelect Configuration Mode
and clickNext
- No changes under
Pre-Build Action
, clickNext
- Set the
Clean Command
toninja -C build_debug clean
- Set the
Build Command
toninja -C build_debug uxas
and clickNext
- No changes under
Source Files
, clickNext
- No changes under
Code Assistance Configuration
, clickNext
- Change
Project Name
toUxAS
and clickFinish
For Linux systems, Netbeans will automatically use the gdb
debugger. On Mac OS X,
gdb
must be installed and signed (see Neil Traft's guide).
If you ever feel the need to refresh external dependencies, you'll need to remove both the downloaded files and the expanded directories:
./rm-external
This script depends upon the presence of the patch tarballs installed
in the /3rd
directory by ./prepare
.
In porting the UxAS build system to Meson/Ninja, we've taken advantage of
wrap
facility to import and build 3rd-party libraries. The advantage
of this approach is that the main UxAS repo no longer needs to contain
these libraries.
There are some rough edges. The wrap
facility (as of April 2017) was
designed to store the necessary metadata on a server operated by the
Meson/Ninja maintainers. There's a very short list of wrapped projects
available from this server. Worse, the wrap
facility is not properly
designed for project-local use: "patches" (often, only the necessary
meson.build
file) are downloaded by the wrap
facility, which offers
no provision for relative URLs.
Furthermore, the patch
file must be in an archive format. This means
that the wrapped project's meson.build
file must be tarred (actually,
the wrap
facility will handle other archive formats) for reference
from the project's wrap file, and the wrap file must contain a valid
SHA256 hash of the patch archive file.
Clearly, this will complicate maintenance. On the plus side, once an external project is properly wrapped, it shouldn't require further work unless you require a different version of the project.
We've taken the approach of stashing valid meson.build
files in the
3rd/wrap_patches
directory. This is the place to store other patched
files (if any) needed for the build of the external project. Note that
"patch" does not refer to a context or unified diff, but rather to an
archive containing new and changed files that overwrite the unzipped
sources. The wrap
facility is not able to patch using diff files.
Boost is handled slightly differently from the other external
dependencies, in that the build system attempts to use a
system-provided version of Boost before falling back on the wrap
facility as a last resort.
Boost uses a bespoke configuration and build system
that is very difficult to replicate with a Meson-based wrap
build,
and so Meson itself handles Boost differently from other
pkg-config
-provided system dependencies.
We strongly recommend using a system-provided Boost from brew
,
apt-get
, etc. If you have a system-provided boost, but during
Meson's configuration phase, you see something like the following, try
setting your BOOST_ROOT
environment variable to the prefix of your
system-installed packages (most likely /usr/local
for MacOS with
Homebrew):
Dependency Boost (filesystem, regex, system) found: NO
If you have a system-provided Boost but this message still does not go away, open an issue with details of your system configuration.
If no system-provided Boost is available, Meson will fall back to
using the wrap
we maintain alongside the other external
dependencies. This will probably work on 64-bit Linux systems, but
unexpected trouble may arise on other platforms.
- Assuming that in the file system, OpenAMASE is at the same level as
OpenUxAS
- Add python package for UxAS plotting (src/Utilities/localcoords)
sudo -H python3 setup.py install
- Run examples
- Example 2: Follow README.md in
examples/02_Example_WaterwaySearch
- Example 3: Follow README.md in
examples/03_Example_DistributedCooperation
- Example 2: Follow README.md in
Building the Documentation on Ubuntu Linux / Bash on Ubuntu on Windows -or- Mac OS X (Partially-Automated)
The following is a bash script that helps to partially-automate the "building the documentation" processes that are documented in this README.md file below.
This has been tested-working on Ubuntu 16.04, as of 2017-05-23.
- Download the script from the OpenUxAS repository (build_documentation.sh) OR
cd
to yourgit clone
d OpenUxAS directory - Run the script at the terminal:
./build_documentation.sh
- Follow the on-screen instructions
Note that this will pop open two html files in your webbrowser and also the pdf manual when run.
If you'd like to do this process manually, then:
- The User Manual can be generated by running:
pdflatex UxAS_UserManual.tex
in the folderdoc/reference/UserManual/
- Create HTML Doxygen reference documenation:
- Open terminal in directory
doc/doxygen
sh RunDoxygen.sh
- In newly created
html
folder, open index.html
- Open terminal in directory
- Doxygen PDF reference manual can be created by:
- Copy the line from
ExtraLineToFixLatex.txt
intodoc/doxygen/latex/refman.tex
just above the line%===== C O N T E N T S =====
- In the folder
doc/doxygen/latex
run the commandpdflatex refman.tex
- The complete reference manual can be found at
doc/doxygen/latex/refman.pdf
- Copy the line from
The OpenUxAS branching model addresses the following concerns:
- We have a stable branch that always builds and passes tests
- Multiple collaborative teams can proceed with their development independently
- Discrete features can be contributed to the main line of OpenUxAS development, and these can be integrated into other teams' ongoing work
- Until OpenUxAS is public, all teams can use the
afrl-rq
organization's Travis-CI account for continuous integration
To address these concerns, OpenUxAS uses a variant on the Git Flow, GitLab Flow, and GitHub flow models.
Because OpenUxAS does not yet have a fixed cycle of releases, we do
not need the additional complexity of hotfix/
and release/
branches present in Git Flow. However, since a number of collaborating
teams work on OpenUxAS simultaneously, it makes sense to have
long-lived branches for each team, rather than only having feature
branches and a stable branch.
This README does not go into detail about the various Flow models, but instead provides instructions for common scenarios. We encourage you to read about the Flow models to get more of a sense for the "why"; here we are focusing on the "how".
The repository will typically have a branching structure like the following:
master
- very stable, only updated by pull request from
develop
- very stable, only updated by pull request from
develop
- stable, only updated by pull request from feature branches
teamA
- team branch for Team A
- stable at the discretion of Team A
- updated by merging in feature branches and
develop
teamA-feature1
- feature branch for Team A
- when finished, merged into
develop
via pull request
teamB
teamB-feature1
- etc.
The team branch is the branch off of which your team will work. It
serves the role of the develop
branch of Git Flow or the master
branch of GitLab and GitHub Flow. This branch is never intended to be
directly merged back into develop
, but feature branches based off of
it will be.
If you have experience with these models, this concept probably seems
odd. Eventually, we would like to replace these team branches with
entire repo forks for each team, but until OpenUxAS is public, this
would prevent forks from using the afrl-rq
Travis-CI account.
Start by creating a new branch that will serve as the active development branch for your team. This step should only be necessary once for your team; this branch is meant to be long-lived as opposed to a feature branch that is quickly merged in and deleted.
$ git checkout develop
$ git checkout -b teamA
You will want to regularly incorporate the latest changes from the
develop
branch in your team branch. This reduces the pain when
merging your team's changes back into develop
.
Start by making sure your local develop
branch is up-to-date:
$ git checkout develop
$ git pull
Then merge the updated develop
with your team branch:
$ git checkout teamA
$ git merge develop
Feature branches are shorter-lived branches meant to encompass a
particular effort or feature addition. These branches will be the
means for you to incorporate your team's changes into the main
develop
branch via pull requests.
Feature branches will always be based off of your team branch, so if
your team branch has commits you would like to see in develop
, you
can simply create a new feature branch and begin the pull request
process right away.
To help the OpenUxAS maintainers know which branches belong to which
teams, feature branches should be named using your team name as a
prefix, for example teamA-feature1
.
Create a feature branch by checking it out off of your team
branch. Note that it will save you some effort at the later merge to
update your team branch from develop
first.
$ git checkout teamA
$ get checkout -b teamA/feature1
For a long-running feature branch, you may want to occasionally merge
it back into your team branch so it can be shared within your team
before it's ready to be merged into develop
.
$ git checkout teamA
$ git merge teamA/feature1
You cannot directly merge a feature branch into develop
, because it
is protected. Instead, open a pull request from the feature branch
into develop
, and your changes will be merged after review.
It is a good idea to update your team branch and delete your feature
branch once it is merged into develop
.
$ git checkout develop
$ git pull
$ git checkout teamA
$ git merge develop
$ git push origin --delete teamA/feature1
$ git branch -d teamA/feature1