NeuronBridge backend services

Deployment

As a prerequisite, you need to have the AWS CLI installed and configured with proper AWS credentials.

Deploy NeuronBridge shared VPC

The VPC deployment is a one time deployment for ALL deploy environments, no matter whether they are DEV, PROD, TEST, VAL and so on. This will create a VPC that will be shared by all alignment jobs so be careful not to remove it because if you do you will have to redeploy all align environments. To deploy:

cd vpc
npm install
npm run sls -- deploy

Notice that stage is not needed because it will deploy to a 'shared' stage ('janelia-neuronbridge-vpc-shared')

Deploy NeuronBridge compute alignment stack

NeuronBridge compute alignment requires an AMI instance preconfigured with ECS and with all required volumes mounted as expected by the alignment batch job.

To create the AMI use these steps:

start an Amazon ECS-optimized Amazon Linux AMI.
start the EC2 instance
run the following commands that mount the expected volumes:

sudo yum -y update
sudo yum install -y fuse-devel
sudo mkfs -t ext4 /dev/xvdb
sudo mkdir /scratch_volume
sudo echo -e '/dev/xvdb\t/scratch_volume\text4\tdefaults\t0\t0' | sudo tee -a /etc/fstab
sudo mount –a
sudo stop ecs
sudo rm -rf /var/lib/ecs/data/ecs_agent_data.json

save an image from the running EC2 instance

Once the AMI instance ID is available make sure you set the proper AMI instance in align/serverless.yml. To deploy:

cd align
npm install
npm run sls -- deploy -s dev

The command above will create the compute environment, the job definition and the job queue.

Deploy NeuronBridge¸ color depth search stack

Make sure you have built all the java packages with jdk 1.8 and maven:

mvnw clean package

In order to create the color depth search lambdas run:

cd search
npm install
npm run sls -- deploy -s dev

To deploy with different search limits:

PER_DAY_SEARCH_LIMITS=2 CONCURRENT_SEARCH_LIMITS=2 npm run sls -- deploy -s cgdev

Note: a negative value for a limit means unlimited.

To update a single function, once you have a deployed stack

npm run sls -- deploy function -f <function_name> -s dev

Alignment Parameters

force_voxel_size - if true it uses resolution parameters provided by the user
xy_resolution
z_resolution
reference_channel
number of slots (not exposed to the user)

Color Depth Search Parameters:

dataThreshold - default: 100
maskThreshold - default: 100
pixColorFluctuation (zSliceRange) - default: 2
xyShift - default: 2
mirrorMask - default: true
minMatchingPixRatio - default: 2