OpenCGA
Page tree
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

OpenCGA uses ARM template to auto deploy a pool with preconfigured opencga docker image. This pool is "AutoScale" enabled so will scale as number of variant index jobs will grow. These ARM script will also populate the following section in "configuration.yml"  which are enables OpenCGA daemon to submit job (azure task) to Azure Batch Service.

Configuration 

configuration.yml
....
execution:
  mode: AZURE
	...
  options:
    #Azure Batch Service information
    batchAccount : "batchAccount"
    batchKey : "batchKey"
    batchUri : "https://batchservice.uksouth.batch.azure.com"
    batchPoolId : "poolId"
    dockerImageName : "openCGADockerImageName" # preconfigured docker image
    dockerArgs : "dockerRunOptions"   # e,g; mount points etc.
....

Variant Index Job Creation

Once user create an OpenCGA variant indexing job, this will be stored in OpenCGA catalog. For example, following is an example to link a file in catalog and create index pipeline which internally will be stored as a catalog job :


OpenCGA Variant Index Job Creation

./opencga.sh files link -i ALL.chr22.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz -s "sudy" 
./opencga.sh variant index --file ALL.chr22.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz --calculate-stats --annotate -o tmp

Once OpenCGA daemon is running, will fetch available jobs from the catalog, prepare them and then submit each catalog job as an "Azure Task" to the pool specified in "configuration.yml". A typical Azure task command will look like :


Azure Task Command
/opt/opencga/bin/opencga-analysis.sh variant index --outdir /opt/opencga/sessions/jobs/J_2510 --merge ADVANCED -DaggregatedType=NONE -DcalculateStats=true --annotate  -Dstdin=false -Dstdout=false --file ALL2.chr22.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz -DoverwriteAnnotations=false -Dsid=eyJkjGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0IiwiYXVkIjoiT3BlbkNHQSB1c2VycyIsImlhdCI6MTU0Njk0OTM3NiwiZXhwIjoxNTQ2OTUyOTc2fQ.oRKCs0uUANRwZOp12NxbY3st6MVe9K1Wp3eMH1Bgjdc -Dinclude.extra-fields=all -DpostLoad.check.skip=false -Dload.split-data=false -Dexclude.genotypes=false --path tmp:


On startup, docker container will mount the locations listed in "dockerArgs"  parameter in "configuration.yml" file, e.g; "/opt/opencga/conf", "/opt/opencga/sessions", "storage location" (where variant files are stored) and any other run time options. 


  • No labels