A hybris® build and deployment framework
ydeploy is a continuous build and deployment framework for the hybris commerce suite.
ydeploy integrates with the hybris platform build framework and attempts to reduce the amount of effort necessary to implement automated build and deployment tasks for hybris software across your environment landscape. Like the hybris build framework, ydeploy is built on Apache Ant and should be easy to pickup for most Java developers.
ydeploy is intended to be a simple, yet extensible tool.
The standard distribution provides a set of configurable build targets intended to address the most common build and deployment requirements. When necessary, the framework can be customized and extended to meet various other operational tasks related to administering hybris software.
At the most basic level, the ydeploy framework provides you a simple way to perform the following tasks:
- Build and generate "build once, deploy everywhere" hybris release packages
- Deploy your hybris release packages out to any number of servers or environments
- Perform standard hybris server administrative (e.g. start, stop, restart) tasks from your build server.
However, if you dig in a bit deeper you will find that ydeploy actually provides you with quite a bit more.
- Quick and easy setup of hybris build & deployment jobs from your continuous integration server
- Support for starting, stopping, and restarting your hybris servers from your build server
- Ability to execute multiple build, release, and deployment jobs concurrently from a single build server
- Support for automated deployment and installation of the hybris platform software on new servers
- Support for automated deployment of hybris platform upgrades on existing servers
- Ability to utilize different hybris platform release versions across your development branches and environments
- Framework is platform agnostic and works across most Windows, Linux, and Unix variants
- An optimal approach to manage and deploy your global, environment specific, and application instance specific hybris property configurations.
- Ability to dynamically derive or discover your hybris configuration values at build or deploy time.
- Automatic packaging and deployment of your custom server configuration files (tomcat, tcServer, solr), hybris License files, or platform customization files.
- Many other similar administrative & operational tasks can be simplified by implementing them on the ydeploy framework.
ydeploy is not a comprehensive all-in-one solution intended to address all of your hybris continuous deployment needs. There are many very capable general-purpose tools available which can be used to address individual parts of your deployment strategy. ydeploy is intended to be an open tool that you can use to fill the functional gap between these general-purpose tools and the limited continuous deployment capabilites that the hybris plaform provides out of the box.
Below is a sampling of capabilities that ydpeloy does not provide natively. We leave it up to you to decide the best tool to meet these needs for your organization.
- Continuous Integration Server: While ydeploy orchestrates a lot of the activities that happen after kicking of a build task, we highly recommend using a CI server such as Jenkins, AntHill, or Bamboo (to name a few) to provide a simple interface for kicking off your ydeploy builds, managing build lifecycles, addressing authentication & access control needs, etc.
- Source Control: ydeploy is not a source control tool and does not itself checkout code, tag releases, or perform any other source control related operations. Most CI servers have very strong capabilities releated to source control needs, so we recommend using these capabilites when available.
- Artifact Repository: ydeploy does not itself manage or archive your build artifacts. CI server typically provides these capabilities or will have integration with such solutions.
- File transfer: When performing deployments, ydeploy does not actually distribute (i.e. copy) release artifacts out to target server. Again, most CI servers typically have very strong capabilities to copy files and invokes processes on remote servers in your environment landscape.
Before diving into the low level details regarding how to setup ydeploy, you may be wanting to see a more concrete example regarding what the ydeploy tool can do for you. In this section we'll show you the standard capabilities of the solution, but gloss over the fine print regarding how you can customize and configure the tool to meet your needs. For detailed information regarding integrating ydeploy into your hybris solution, please see the "Using ydeploy" section.
ydeploy is designed to be checked into your source control repository along with your hybris custom extensions and configuration directories.
hybris-repo
/config
/localextensions.xml
[ . . . ]
/custom
/custext1
/custext2
[ . . . ]
/ydeploy
After checking out the contents of your repo, we can use ydeploy to perform a full build of your custom hybris extension using the build command.
$ ls
config custom ydeploy
$ cd ydeploy
$ ant build
Buildfile: /opt/build/workspace/ydeploy/build.xml
[echo] Initializing ydeploy
[ . . . ]
BUILD SUCCESSFUL
Total time: 49 seconds
That's it! ydeploy has successfully performed a full build of your hybris solution for you.
You can see the built solution by looking into the ydeploy/temp/build directory.
$ cd ydeploy/temp/build/hybris
$ ls
bin config data log sampleconfigurations temp
At this point you may be scratching your head and asking "why not just invoke the hybris build directly from the hybris/bin/platform directory?"
Great question! To answer, lets think through all of the steps we would have had to perform to achieve the same thing without ydeploy.
- First, we would need to have the hybris software setup on the machine beforehand. In the example above, hybris was not even setup on the machine prior to running the ydeploy
buildcommand. ydeploy automatically set everything up - After setting up hybris we would need to deploy the custom extensions and config directories from the checkout location to the proper locations within the hybris install
- Next, we would likely need to update the local.properties, localextensions.xml and other configuration files within the hybris/config directory to ensure the build process runs correctly on this particular machine.
- Almost ready to build. But first, we would need to go into the hybris/bin/platform directory and setup the environment using that
setantenv.shscript - At this point we should be able to run the standard hybris build process
Clearly ydeploy has helped simplify the process. Next, lets look at how to generate a release using ydeploy.
In this section we'll look at how to build and generate a hybris release package using ydeploy. The release packages generated by the genrelease command encapsulate all of the software and configuration required to deploy hybris across your environment landscape. This includes the hybris platform software, any custom extensions, and the hybris server configuration.
Similar to the build task, genrelease can be done in a single command
$ cd ydeploy
$ ant genrelease
Buildfile: /opt/build/workspace/ydeploy/build.xml
[echo] Initializing ydeploy
[ . . . ]
BUILD SUCCESSFUL
Total time: 1 minute 18 second
Just as with the build command, the genrelease command has setup hybris on the fly, performed the build, and generated a release package which can be used to deploy hybris to any of our environments.
You can see the generated release package by looking into the ydeploy/temp/genrelease directory.
$ cd ydeploy/temp/genrelease
$ ls
hybrisServer-Release-0.zip
The deployrelease command is used to deploy the hybris release package which is generated by the genrelease command to servers in your environment landscape. Specifically, ydeploy performs the following steps:
- Detects the location of the hybris platform on the target server (if installed) and stops the server (if running).
- Extracts the contents of the hybris release packages and deploys them to proper hybris installation location.
- Deploys the proper server specific configuration for the target server and invokes the hybris build.
- Starts the hybris build server on the new release.
Note: hybris does not actually need to already been installed. ydeploy will automatically install and configure the hybris platform as part of the deployrelease task.
One thing to note is that ydeploy isn't designed to actually copy the hybris release package from your build server to the target server, but once the package is copied to the destination server, ydeploy can handle the rest.
Note: To copy the hybris release package from your build server to your target server, we recommend using the capabilities of your continuous integration server. Further discussions regarding how to integrate ydeploy into Jenkins can be found in the Integrating ydeploy with Jenkins section.
For the example below, I've copied the hybrisServer-Release-0.zip which we generated above to the target server. As the ydeploy package is embedded into the hybris release package, we first need to extract the zip before we can invoke ydeploy.
$ cd /opt/build/releases
$ ls
hybrisServer-Release-0.zip
$ unzip hybrisServer-Release-0.zip & cd ydeploy & ant deployrelease
Buildfile: /opt/build/releases/ydeploy/build.xml
[echo] Initializing ydeploy
[ . . . ]
BUILD SUCCESSFUL
Total time: 2 minutes 25 second
At this point our new release has been deployed to the target server.
In this section we will provide a quick walk through of how to integrate ydeploy into your hybris solution.
You can download a copy of the current ydeploy release using the master branch of the Git repository. You can either clone the repo using your Git client of choice or you can download a zip of the repo content using the link below.
https://github.com/avatria/ydeploy/archive/master.zip
Before we get started with integrating ydeploy into your hybris solution, lets do a quick review of the contents of the distribution.
Below is the basic directory structure of the ydeploy software.
ydeploy/
/core
/conf
/core.properties
/resources
/custom
/conf
/ydeploy.properties
/env
/resources
/template
/hybris
/config
/buildcallbacks.xml
/ydeploy
The ydeploy/core directory contains configuration and resources which are integral to the framework and typically should not be edited.
The ydeploy/custom directory should contain all user specific configurations and customizations. The intent is to keep user specific customizations separate from core framework code and configuration. ydeploy.properties and the env directory will be where most of your configuration customizations will reside.
The ydeploy/template directory contains files and sample configurations that are intended to copied in your standard hybris install structure. Currently all template files should be placed under your hybris/config directory (i.e. ${HYBRIS_CONFIG_DIR}).
It is recommended that you add the full contents of the ydeploy distribution to your source code repository right along side your hybris customizations directory (e.g. custom) and your hybris/config directory.
Additionally, you should copy the contents of the ydeploy/template/hybris/config into your hybris/config directory. This step is necessary to allow ydeploy to hook into certain behaviors of the hybris build framework.
Below is an example of how your source control repo contents may look after integrating ydeploy.
hybris-repo
/config
/buildcallbacks.xml [from ydeploy distribution template dir]
/ydeploy [from ydeploy distribution template dir]
/custom
/custext1
/custext2
[ . . . ]
/ydeploy [full ydeploy distribution]
Once you have copied ydeploy into your source control repository, you will need to review and edit some of the common configurations. Below are the primary configuration you should review and edit.
These are the ydeploy framework configurations that likely may need editing.
# Where your hybris 'custom' extensions and 'config' folders are stored in source control in relation
# to the ydeploy directory.
yd.hybris.custom.ext.dir=${yd.root.dir}/../custom
yd.hybris.custom.config.dir=${yd.root.dir}/../config
# hybris build tasks to perform when the ydeploy 'build' target is invoked
yd.tasks.build.hybrisbuildtargets=customize all
# hybris build tasks to perform when the ydeploy 'genrelease' target is invoked
yd.tasks.genrelease.hybrisbuildtargets=customize all production
# hybris build tasks to perform when the ydeploy 'deployrelease' target is invoked on the target server
yd.tasks.deployrelease.hybrisbuildtargets=all
This file contains your global ydeploy framework configurations.
# Where hybris platform zip releases are stored
yd.hybris.platform.releases.dir=/opt/build/hybris/releases
# The hybris platform zip file to use for 'build' and 'genrelease' targets.
# Must exist in the ${yd.hybris.platform.releases.dir} location
yd.hybris.platform.release.file=hybris-commerce-suite-5.4.0.1.min.zip
# Location at which the 'deployrelease' and server related targets will operate.
yd.hybris.platform.deploy.dir=/opt/hybriscs
The first two settings described above (yd.hybris.platform.releases.dir, yd.hybris.platform.release.file) are very important and tell ydeploy where to find the standard hybris platform release zip to use to setup hybris 'on the fly' for the build and genrelease commands. You must configure these properties to point to the hybris release ZIP file you would like to use.
Note: For hybris releases prior to 5.5, the release ZIP file can be the vanilla hybris install which you downloaded from the hybris/SAP site. If you are using hybris 5.6 or later it is recommended that you combine the hybris commerce suite ZIP with the hybris Maven dependencies ZIP (e.g. hybris-dependencies), as these dependencies may be required if you are using archetypes or modules leveraging Core+.
The last setting (yd.hybris.platform.deploy.dir) tells ydeploy where to install hybris on your server when the deploy command is executed.
The configuration values set in global.properties will be used unless you choose to provide different configurations to be used at an environment or server specific level.
ydeploy
/custom
/conf
/env
/global.properties
/dev
/dev.properties
/dev01.properties
/dev02.properties
/qa
/qa.properties
When you invoke ydeploy it will determine what configurations to use based upon any environment or server specific configurations you provide. This is described more in the next section.
When you invoke ydeploy, it will first look to detect any environment or server specific configuration values which you may have provided via environmental variables or passed in via command line parameters.
- Environmental Variables
HYBRIS_ENVHYBRIS_SERVER_ID
- Command Line Parameters
-Dhybris_env-Dhybris_server_id
Typically, we recommended only using the command line parameter approach for initial testing and debugging purposes.
Below is an example of running ydeploy using the command line argument approach.
$ cd ydeploy
$ ant build -Dhybris_env=dev -Dhybris_server_id=dev01
In this example, the effective ydeploy framework configuration will be loaded in the following order with later configuration values taking precedence.
- ydeploy/custom/conf/env/global.properties
- ydeploy/custom/conf/env/dev/dev.properties
- ydeploy/custom/conf/env/dev/dev01.properties
Once you have all of the configuration layers setup properly, we recommend setting up the environmental variables across all of the build and target hybris application servers that ydeploy will run. This will allow for the ydeploy framework to infer the proper configurations to use from the environment itself such that the command line parameters can be omitted.
One of the key benefits of ydeploy is that it provides the ability to manage hybris Configuration Properties in an optimal manner by allow you to define your configuration values at at any one of three levels.
- Global Configuration Properties
- Environment-specific Configuration Properties
- Application Instance-specific Configuration Properties
These configuration levels should reside within the ${HYBRIS_CONFIG_DIR}/ydeploy/localproperties directory. An example structure is provided below but is also available in the ydeploy distribution at the ydeploy/template/hybris/config location.
/config [ ${HYBRIS_CONFIG_DIR} ]
/ydeploy
/localproperties
/global.properties
/dev
/dev.properties
/dev01.properties
/dev02.properties
/qa
/qa.properties
/qa01.properties
/qa02.properties
[... etc ...]
Whenever a hybris build is invoked, ydepoy will merge the three levels of properties files together in the order specified above to generate the effective local.properties at the ${HYBRIS_CONFIG_DIR}/local.properties location. When a property value is is defined at multiple levels, later levels override earlier, more global levels.
You will find that this flexible structure allows for you to greatly reduce duplication of local.properties configurations which ultimately helps reduce maintenance and risk of errors or omissions.
global.properties refers to the properties file that contains configuration that will apply to every environment and every node in every environment unless that property is overridden at a lower level (i.e. environment or environment node).
<ENV>.properties refers to a Properties file that contains the configuration specific to an entire environment. The file must be named using the convention /.properties name where is the Environment ID that is supplied to the ydeploy framework via an environment variable (${HYBRIS_ENV_ID}) or passed to ydeploy via a command line parameter (-Dhybris_env_id=<ENV>). This Environment ID must be the name of the folder that the Environment-specific Configuration Properties reside in and must also be the name of the file that contains the Environment-specific Configuration Properties. e.g. Given an Environment ID of dev, the following must exist in the config directory.
<SERVER_ID>.properties refers to a Properties file that contains the configuration instance-specific configuration for an individual hybris JVM. The file must be named using the convention /<SERVER_ID>.properties name where is the Environment ID which the server belongs, and <SERVER_ID> represents the unique identifier for the hybris JVM process. These variables are supplied to the ydeploy framework via environment variables (${HYBRIS_ENV_ID} and ${HYBRIS_SERVER_ID}) or passed to ydeploy via a command line parameters (-Dhybris_env_id=<ENV>, -Dhybris_server_id=<SERVER_ID>).
If you are performing clean hybris installs often using ydeploy, you may find that a significant amount of the time is actually spent just extracting the hybris release ZIP file prior to running the build. If you find the extraction of the full hybris release ZIP is consuming a lot of time, you can typically speed things up significantly by stripping out files or directories from the ZIP which you are not using. Unused hybris extensions, such as accelerator templates and sampledata, and the hybris-Mobile-Apps-SDK are usually pretty good candidates for removal. Basically, make the ZIP file as small so that the unzip goes quicker.
ydeploy was originally developed by Avatria, Inc. (http://www.avatria.com) and donated to the community as an Open Source project under the Apache 2.0 License. The project continues to be maintained by the Avatria team, however, contributions and involvement by all organizations and individuals is welcome.
For issues, suggestions, or feature requests, please post an issue (https://github.com/avatria/ydeploy/issues).
For comments, questions, or feedback please send a message to @zmcmahon-avatria.
hybris is the trademark of SAP SE in Germany and in several other countries. ydeploy is an independently developed open source software solution which is maintained and supported by the ydeploy community and distributed under the Apache 2.0 License. ydeploy was not developed by hybris or SAP.