Deployment

OrderFlow Deployment

Assuming that an environment has been set up as previously described, OrderFlow instance(s) can be deployed by following the instructions in this section.

New OrderFlow Installation

Before OrderFlow can be deployed, there are some scripts and configuration files that need to be installed and edited. This process is described in the following section.

Bootstrapping Deployment Scripts

This process starts with downloading the "Deployment Scripts Bootstrap" Ant build file. This is normally done as the rtdadmin user in its home directory:

cd ~
mkdir deploy
cd deploy

curl -u build -o build.xml https://downloads.realtimedespatch.co.uk/general/build.xml

The password for downloads.realtimedespatch.co.uk is available in RemoteMan.

This will result in the following Ant build file:

/home/rtdadmin/deploy/build.xml

Once downloaded, run the following command to install the deployment environment (from the same directory):

ant install

This downloads and unzips the deployment scripts into the deploy-host-jetty subdirectory:

  ~/deploy$ ls -1 deploy-host-jetty/build*
  deploy-host-jetty/build.properties
  deploy-host-jetty/build.xml
  ~/deploy$

Note

In some circumstances, it may be necessary to manually install the certificate for downloads.realtimedespatch.co.uk, as follows:

curl -u build -o InstallCert.jar https://downloads.realtimedespatch.co.uk/general/InstallCert.jar java -cp InstallCert.jar InstallCert downloads.realtimedespatch.co.uk java -cp InstallCert.jar InstallCert remote.realtimedespatch.co.uk

Deployment Properties

All the local configuration for the deployment of OrderFlow is held in the deploy-host-jetty/build.properties file. This file should be checked before every deployment command is invoked, to ensure that the properties within are correct.

A typical build.properties file looks like this:

  download.dir=../downloads
  download.user=********
  download.password=********
  download.fetch.base.url=https://downloads.realtimedespatch.co.uk

  module.base.uri=https://remote.realtimedespatch.co.uk/web
  module.user=********
  module.password=********

  #Uncomment for Linux
  application.base.dir=/usr/share 

  #Uncomment for Windows
  #application.base.dir=/RealtimeDespatch/

  app.name=orderflow
  customer.name=rtd
  instance.name=test

Each property is expanded upon in more detail here:

Property Name	Example Value	Usage
download.dir	../downloads	The directory (relative to deploy-host-jetty) into which downloaded files are placed.
download.user	downuser	The (Basic Authentication) user name submitted to the download.fetch.base.url when downloading files.
download.password	downpass	The (Basic Authentication) password submitted to the download.fetch.base.url when downloading files.
download.fetch.base.url	\https://downloads.realtimedespatch.co.uk	The location from which to download core OrderFlow files.
module.base.uri	\https://remote.realtimedespatch.co.uk/web	The location from which to download optional module files.
module.user	moduser	The (RemoteMan) user name submitted to the module.base.uri when downloading files.
module.password	modpass	The (RemoteMan) password submitted to the module.base.uri when downloading files.
application.base.dir	/usr/share	The (absolute) directory under which OrderFlow is to be deployed.
app.name	orderflow	The application name of OrderFlow, usually left at its default value.
customer.name	rtd	The short (lower-case) customer name that the instance is for.
instance.name	test	The OrderFlow instance name, to distinguish it from other instances.

Note that the last three properties listed above are used to construct the database name, to which the OrderFlow instance will connect. For example, the example values above would result in the application attempting to connect to a database called orderflow_rtd_test.

Deployment Scripts Update

If the deployment scripts need updating, for example, if a new version is available, then the following command (as rtdadmin, not via sudo):

ant update

This command backs up the deploy-host-jetty/build.properties file (which may have modified), replaces the deployment scripts folder with the latest from downloads.realtimedespatch.co.uk, then restores the backed up properties file.

Installation With Database

If the database is installed on the application server, then the OrderFlow application and database schema can both be installed at the same time.

To install OrderFlow and its database, navigate to /home/rtdadmin/deploy/deploy-host-jetty/ and run the following command:

sudo ant install-with-db

Note that this may require sudo privilege as it creates directories and writes files to below the application.base.dir, which is usually owned by root.

The script will prompt for several parameters:

• Target revision - this will be the revision in the name of the built artefact, as found on the Downloads_ server, e.g. for the zip file called orderflow-45561-jetty.zip, revision 45561 should be entered.
• Checkout path - this is the path relative to \https://downloads.realtimedespatch.co.uk/orderflow/ in which the built artefact can be found. This depends on whether a tagged, branch or trunk build is being deployed. Examples include tags/3.6.8-lowestoft_from_45560 or branches/3.6.7-london_from_44661. (If no checkout path is supplied then a trunk build is assumed.)
• Database user - the database user needs to have sufficient privilege to make schema changes on the database, so typically rtdadmin or sometimes root is used here. (Note that the OrderFlow application accesses the database using a less-privileged user, as defined in config/jdbc.properties found under the application.base.dir.)
• Database password - this is the password for the user previously supplied. The appropriate values for these parameters can be found in RemoteMan.

Note

The install-with-db installation target performs the following tasks:

• Asks for confirmation that the database should be installed. This requires you to type true and hit return, to prevent accidentally overwriting any existing database.
• Checks that the instance is not already installed.
• Creates a database, named from the values in build.properties accordingly.
• Creates a database admin user (called rtdadmin by default), and grants suitable privileges to this and the user that OrderFlow will use. (This is a shorter name derived from the database name - e.g. ord_rtd_test.)
• Downloads the core OrderFlow zip and database files (from under the download.fetch.base.url). These contain the core Java application and all necessary scripts, property files, commisioning data etc.
• Downloads any non-core module files available to the customer (from under the module.base.uri).
• Copies (and expands) the downloaded files and modules to the application.base.dir.
• Copies the expanded application start and init.d scripts to the application base and /etc/init.d/ directories respectively.
• Copies configuration files to the config directory below the application.base.dir.
• Loads the downloaded commissioning data into the application's database, and rolls the database forward to the application's version.

One remaining installation task to perform is the installation of database triggers, which (until #9369 has been implemented) must be done manually, by running the following commands (with database name and username substituted accordingly):

cd /tmp
curl -u build -o audit_product_inventory.sql https://support.orderflow-wms.co.uk/svn/orderflow/trunk/rtd2/database/triggers/audit_product_inventory.sql
curl -u build -o drop_all_triggers.sql https://support.orderflow-wms.co.uk/svn/orderflow/trunk/rtd2/database/triggers/drop_all_triggers.sql
curl -u build -o report_configuration_audit_delete.sql https://support.orderflow-wms.co.uk/svn/orderflow/trunk/rtd2/database/triggers/report_configuration_audit_delete.sql
curl -u build -o report_configuration_audit_update.sql https://support.orderflow-wms.co.uk/svn/orderflow/trunk/rtd2/database/triggers/report_configuration_audit_update.sql
curl -u build -o report_design_audit_delete.sql https://support.orderflow-wms.co.uk/svn/orderflow/trunk/rtd2/database/triggers/report_design_audit_delete.sql
curl -u build -o report_design_audit_update.sql https://support.orderflow-wms.co.uk/svn/orderflow/trunk/rtd2/database/triggers/report_design_audit_update.sql
mysql -urtdadmin -p orderflow_rtd_test < drop_all_triggers.sql
mysql -urtdadmin -p orderflow_rtd_test < audit_product_inventory.sql
mysql -urtdadmin -p orderflow_rtd_test < report_configuration_audit_delete.sql
mysql -urtdadmin -p orderflow_rtd_test < report_configuration_audit_update.sql
mysql -urtdadmin -p orderflow_rtd_test < report_design_audit_delete.sql
mysql -urtdadmin -p orderflow_rtd_test < report_design_audit_update.sql

Installation Without Database

If database is on a remote system to which you don't have root access, then it might make sense to install the database separately.

To install OrderFlow (without installing the database), navigate to /home/rtdadmin/deploy/deploy-host-jetty/ and run the following command:

sudo ant install-no-db

Note that this requires sudo privilege as it creates directories and writes files to below the application.base.dir, which is usually owned by root.

The script will prompt for several parameters:

• Target revision - this will be the revision in the name of the built artefact, as found on the Downloads_ server, e.g. for the zip file called orderflow-45561-jetty.zip, revision 45561 should be entered.
• Checkout path - this is the path relative to \https://downloads.realtimedespatch.co.uk/orderflow/ in which the built artefact can be found. This depends on whether a tagged, branch or trunk build is being deployed. Examples include tags/3.6.8-lowestoft_from_45560 or branches/3.6.7-london_from_44661. (If no checkout path is supplied then a trunk build is assumed.)

Note

The install-no-db installation target performs the following tasks:

• Asks for confirmation that the database should not be installed.
• Checks that the instance is not already installed.
• Downloads the core OrderFlow zip and database files (from under the download.fetch.base.url). These contain the core Java application and all necessary scripts, property files, commisioning data etc.
• Downloads any non-core module files available to the customer (from under the module.base.uri).
• Copies (and expands) the downloaded files and modules to the application.base.dir.
• Copies the expanded application start and init.d scripts to the application base and /etc/init.d/ directories respectively.
• Copies configuration files to the config directory below the application.base.dir.

The following commands can be run to install the database separately:

sudo ant db-create
sudo ant db-load

Note that after running the above scripts, the database triggers will still need to be installed as described in the previous section.

Installation Verification

The following command can be run to check that OrderFlow has been installed correctly:

ant check-environment

Starting OrderFlow

OrderFlow can be started in one of two ways. It can be started directly using the instance-specific init.d script, for example:

sudo /etc/init.d/orderflow_rtd_test start

Or the following Ant target can be invoked, from the deploy-host-jetty directory. If this method is used, the build.properties file must be re-checked beforehand, to ensure that the correct instance is being referenced:

sudo ant application-start

Verifying OrderFlow Start-up

The easiest way to verify that OrderFlow has started is by attempting to access its desktop interface from a browser, a minute or two after starting it. However, any problems experienced on start-up will not necessarily be apparent during an ad-hoc navigate around the interface.

Instead, OrderFlow's log files should be inspected during start-up, to verify that no problems have occurred.

The logs directory can be found under the application's base directory, for example in /usr/share/orderflow/rtd/test/logs/. Today's OrderFlow log is called orderflow.log, and today's request log file is called accesslog.log. The OrderFlow log file should be tailed during start-up and watched for any abnormal behaviour. This can be achieved (from anywhere on the file system) by running a command similar to the following:

  tail -f /usr/share/orderflow/rtd/test/logs/orderflow.log

"Abnormal behaviour" to look for would be any of the following:

• Stack traces - lengthy listings of qualified method names, with line numbers, that pinpoint where an error has occurred, e.g.:

  2017-03-22 16:30:37,025 ERROR [DefaultModuleRuntimeManager] Failed to handle loading of application module: rtd2-batch
  org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line 359 in XML document from class path resource [rtd2-batch-context.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 359; columnNumber: 9; cvc-complex-type.2.3: Element 'beans' cannot have character [children], because the type's content type is element-only.
          at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:399)
          at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:336)
          ...
          at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
          at RunJetty.main(RunJetty.java:113)
          at OrderFlow.main(OrderFlow.java:44)
  Caused by: org.xml.sax.SAXParseException; lineNumber: 359; columnNumber: 9; cvc-complex-type.2.3: Element 'beans' cannot have character [children], because the type's content type is element-only.
          at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
          at org.apache.xerces.util.ErrorHandlerWrapper.error(Unknown Source)
          ...

• Modules not loading - this can be detected by the presence of an error in the final list of modules, e.g.:

  2017-03-22 16:30:49,042 INFO [TransitionsLogger] 
  ---------------------------------------------
  Module operations succeeded: false
  Number of operations: 132
    rtd2-api: UNLOADED_TO_LOADED
    rtd2-main: UNLOADED_TO_LOADED
    rtd2-hibernate: UNLOADED_TO_LOADED
    rtd2-dao: UNLOADED_TO_LOADED
    rtd2-process: UNLOADED_TO_LOADED
    rtd2-warehouse: UNLOADED_TO_LOADED
    rtd2-archive: UNLOADED_TO_LOADED
    rtd2-monitoring: UNLOADED_TO_LOADED
  ...
    rtd2-batch: UNLOADED_TO_LOADED
      org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line 359 in XML document from class path resource [rtd2-batch-context.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 359; columnNumber: 9; cvc-complex-type.2.3: Element 'beans' cannot have character [children], because the type's content type is element-only.
  ...

• Scheduler not started correctly (check that the following log is correct - use a live instance):

  2017-03-22 16:47:17,487 INFO [QuartzScheduler] Scheduler meta-data: Quartz Scheduler (v2.1.6) 'DefaultQuartzScheduler' with instanceId 'NON_CLUSTERED'
    Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
    NOT STARTED.
    Currently in standby mode.
    Number of jobs executed: 0
    Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
    Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persistence. and is not clustered.

Stopping OrderFlow

If OrderFlow is required to be stopped for any reason, it can be done so in one of two ways (like start-up). It can be stopped directly using the instance-specific init.d script, for example:

sudo /etc/init.d/orderflow_rtd_test stop

Or the following Ant target can be invoked, from the deploy-host-jetty directory. If this method is used, the build.properties file must be re-checked beforehand, to ensure that the correct instance is being referenced:

sudo ant application-stop

Restarting OrderFlow Automatically

The server environment should be configured to start OrderFlow automatically whenever the server is rebooted. How to do this in the default Ubuntu server environment is covered in the section "Debian Runlevel Configuration Tool"

Note that OrderFlow should not be automatically restarted by external monitoring tools or utilities without this first having been discussed with the OrderFlow support team.

OrderFlow Upgrade

Existing OrderFlow instance(s) can be upgraded by following the instructions in this section.

Manual Steps

Before upgrading any instance of OrderFlow, it is essential to check if there are any Manual Steps to be performed, either before or after the main upgrade steps.

Manual steps are detailed on the following page:

https://support.orderflow-wms.co.uk/projects/matrixsupport/wiki/ManualSteps

To determine which manual steps apply to the current upgrade, the current version of OrderFlow needs to be determined. This can be found from the 'About' box in the running instance, or it can be found on the application server by inspecting the OrderFlow current symbolic link, as shown below:

rtdadmin@customertestnew:~$ ls -l /usr/share/orderflow/rtd/test/ | grep current
lrwxrwxrwx 1 root root   21 Dec 22  2014 current -> orderflow-35384-jetty

Any manual steps that reference revisions above the current version and below the target version must be read to see if they are applicable to the current instance being upgraded.

If applicable, the instructions must be followed. If it is not possible to follow the instructions for any reason, then a ticket should be raised and assigned as 'Awaiting Dev. Input' for actioning as soon as possible.

Upgrade With Database Changes

The most common form of OrderFlow upgrade is an upgrade to a tagged version. This includes database changes that correspond with the software changes, so these need to be applied to the database in the form of roll-forward scripts.

Disable monitoring before the upgrade: https://old.uptimerobot.com/login?rt=https://old.uptimerobot.com/dashboard.php Pause uptime robot

https://remote.realtimedespatch.co.uk/web/config/node/home.htm?menu=config/node Select the instance and click edit, then set the expected status as either blank or No Preference.

Perform a cleanup of old backups and orderflow folders: /usr/share/orderflow/customer/live/old : keep the last 3 files /home/rtdadmin/deploy/dbbackups: keep the last backup

Firstly, navigate to /home/rtdadmin/deploy/deploy-host-jetty/ and check that the build.properties file references the correct instance. Then run the following command from this directory:

sudo sh log.sh ant upgrade-with-db

The script will prompt for several parameters:

• Target revision - this will be the revision in the name of the built artefact, as found on the Downloads server, e.g. for the zip file called orderflow-45561-jetty.zip, revision 45561 should be entered.
• Checkout path - this is the path relative to \https://downloads.realtimedespatch.co.uk/orderflow/ in which the built artefact can be found. This depends on whether a tagged, branch or trunk build is being deployed. Examples include tags/3.6.8-lowestoft_from_45560 or branches/3.6.7-london_from_44661. (If no checkout path is supplied then a trunk build is assumed.)
• Database user - the database user needs to have sufficient privilege to make schema changes on the database, so typically rtdadmin or sometimes root is used here. (Note that the OrderFlow application accesses the database using a less-privileged user, as defined in config/jdbc.properties found under the application.base.dir.)
• Database password - this is the password for the user previously supplied. The appropriate values for these parameters can be found in RemoteMan.

Note

The upgrade-with-db target performs the following tasks:

• Asks for confirmation that the correct instance is to be upgraded.
  This depends upon properties set in the build.properties file.
• Stops the running instance, if required.
• Backs-up the database to ~rtdadmin/deploy/dbbbackups, unless user directs otherwise. Note that it is highly recommended that the database is backed-up, as any bugs in roll-forward scripts have the potential to render the existing database un-usable!
• Compresses the database backup file.
• If the upgrade is to a branch version of OrderFlow, and database changes are being applied (which will be the case is the upgrade is from a previous tagged version of OrderFlow to a newer version that has already been branched), then the script asks for confirmation that this is correct.
• Downloads the core OrderFlow zip and database files (from under the download.fetch.base.url), for the target version concerned.
• Downloads any non-core version-specific module files available to the customer (from under the module.base.uri).
• Copies (and expands) the downloaded files and modules to the application.base.dir.
• Rolls the database forward to the application's version.

Once the upgrade is complete and any relevant manual steps have been applied, it is necessary to restart OrderFlow by using the following command:

sudo ant application-start

Verify that OrderFlow has started successfully by following the instructions in the Verifying OrderFlow Start-up section.

Note that some manual steps may be more easily applied once the application has been restarted.

Ensure you reenable the system monitoring.

Upgrade Without Database Changes

When the upgrade is within a branch of OrderFlow (i.e. between the branch's tag and a revision on that branch, or between two revisions on the same branch), the no database changes will be required.

Disable monitoring before the upgrade: https://old.uptimerobot.com/login?rt=https://old.uptimerobot.com/dashboard.php Pause uptime robot

https://remote.realtimedespatch.co.uk/web/config/node/home.htm?menu=config/node Select the instance and click edit, then set the expected status as either blank or No Preference.

Perform a cleanup of old backups and orderflow folders: /usr/share/orderflow/customer/live/old : keep the last 3 files /home/rtdadmin/deploy/dbbackups: keep the last backup

This kind of upgrade is quicker, as no database back-up is required. To upgrade with no database changes, navigate to /home/rtdadmin/deploy/deploy-host-jetty/ and check that the build.properties file references the correct instance.
Then run the following command from this directory:

sudo sh log.sh ant upgrade-no-db

The script will prompt for the target revision and the checkout path (which will typically be a branches path).

Note

The upgrade-no-db target performs the following tasks:

• Asks for confirmation that the correct instance is to be upgraded.
  This depends upon properties set in the build.properties file.
• Stops the running instance, if required.
• Downloads the core OrderFlow zip and database files (from under the download.fetch.base.url), for the target version concerned.
• Downloads any non-core version-specific module files available to the customer (from under the module.base.uri).
• Copies (and expands) the downloaded files and modules to the application.base.dir.

Again, once the upgrade is complete and any relevant manual steps have been applied, OrderFlow needs to be started:

sudo ant application-start

After verifying that OrderFlow has started successfully, check that all relevant manual steps have been applied.

Ensure you reenable the system monitoring.

Environment Checklist

Software

• In addition to the OrderFlow environment the user 'rtdadmin' should have access to the 'ant','curl' and 'zip' utilties required during the initial deployment and subesquent upgrades.

Access

• The SSH user 'rtdadmin' should have access to the server (ideally on port 91)
• Incoming access for SSH user 'rtdadmin' should be restricted to the OrderFlow source IP addresses (see section "Operating System")
• The SSH user 'rtdadmin' should be given 'sudo' rights (or the permissions described in the section "Operating System")
• The database user 'rtdadmin' should have all rights to the OrderFlow database(s)
• Consider whether browser access to the OrderFlow environment should be restricted to fixed IP addresses

Backups, logs, data archiving and automatic startup

• The OrderFlow database should be included in a backup policy appropriate for mission critical business environments
• The OrderFlow log rotation functionality should be enabled to ensure log files do not continue to grow indefinitely
• Consider whether OrderFlow's data purge and data archive functionality can be used to remove data from the OrderFlow environment when it is no longer of value
• OrderFlow should start automatically when the server is powered up but should not be automatically restarted thereafter without discussion with the OrderFlow support team.