Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Note

New OpenCGA 2.0.0 Feature. Work in progress

Overview

OpenCGA produces four different Docker images for different use cases, these are:

  • opencga-base: base image for building the other images, this image contains the OpenCGA basic installation folder  
  • opencga-rest: image running REST web services
  • opencga-master: image running master service
  • opencga: all-in-one image with all OpenCGA components running

You can use opencga Docker image to run a complete OpenCGA platform locally. You can use the other images to deploy a whole cluster solution in a cloud environment using Kubernetes. Docker Images are deployed in Docker Hub OpenCB organisation.

Design and Implementation

Images

All images run with the user opencga and run Java 8. Docker images are deployed in Docker Hub.

opencga-base

This image contains the basic installation in directory /opt

More info at https://hub.docker.com/repository/docker/opencb/opencga-base

opencga-rest

This image is based in opencga-base and runs REST web services using the REST server command-line.

opencga-master

This image runs the master service

opencga

all-in-one image with all OpenCGA components running 

Implementation

OpenCGA publishes a number of images into DockerHub for user ease. These images are based on Alpine JRE images to keep sizes as small as possible, contains OpenCGA binaries, for complete contents of image, please have a look at Dockerfile in github. A typical image name will follow the following structure :

  • opencga:{OPENCGA_VERSION_NUMBER}-{VARIANT_STORAGE_FLAVOUR}
    • OPENCGA_VERSION_NUMBER  will be like 1.4.0, 1.4.2, 2.0 etc
    • VARIANT_STORAGE_FLAVOUR can be mongoDB 4.0, hdinshigh, emr etc
    • e.g. opencga:1.4.0-mongo4.0


OpenCGA has published a docker image for quick testing and playing without going through hassle to learn, wait and install each and every OpenCGA components. The docker image is available at the public docker registry under the repository opencb/opencga-demo. 

The OpenCGA demo docker image contains the following components:

  • OpenCGA binaries
  • MongoDB 4.0
  • Solr 6.6 (default
  • init.sh, a bash script to :

    ...

      1. Install OpenCGA catalog 

    ...

      1. Populate data (optional)

    To download OpenCGA demo image, use the command docker pull with the OpenCB enterprise, opencga-demo repository and

    ...

    tag. For example

    ...

    :

    Code Block
    languagebash
    themeRDark
    titleDownoload the OpenCGA demo image
    $ docker pull opencb/opencga-demo:1.4.

    ...

    2


    To start the OpenCGA demo container, use the command docker run. For example: (This command will pull image as well if not present in local repository)

    Code Block
    languagebash
    themeRDark
    titleStart the OpenCGA container (without loading demo data)
    $ docker run --name demo -d opencb/opencga-demo:1.4.

    ...

    2


    The first time you run the OpenCGA demo container, it installs the OpenCGA catalog in mongoDB, starts the OpenCGA REST server and finally, it starts the OpenCGA daemon. In this case, no data is loaded. If you want to load some demo data, set the option

    ...

    load to

    ...

    true that instructs OpenCGA demo container to download data from Corpasome project and then, to run an ETL pipeline that loads, annotates, computes statistics and indexes data into the Solr search engine. This process roughly takes 35-40 minutes depending on the internet speed and the machine specifications. Following command starts the OpenCGA container loading demo data:

    Code Block
    languagebash
    themeRDark
    titleStart the OpenCGA container with loading demo data
    $ docker run --name demo -e load=true -d opencb/opencga-demo:1.4.2

    User can also mount customise directories to store MongoDB and Solr data for ease. The following command will mount the ~/data/mongodb and ~/data/solr directories from host machine to inside container at "/data/opencga/mongodb", "/data/opencga/mongodb" respectively. 

    Code Block
    languagebash
    themeRDark
    titleStart the OpenCGA container loading demo data And mount user directories
    $ docker run --name demo -e load=true -v ~/data/mongodb:/data/opencga/mongodb -v ~/data/solr:/data/opencga/solr -d opencb/opencga-demo:1.4.2

    In case, user has previously used mongodb and solr data outside of docker, mounted on host machine and now want to use NEW docker image and force docker to NOT install catalog, use the following command: 

    Code Block
    languagebash
    themeRDark
    titleStart the OpenCGA container loading demo data And mount user directories Without Installing Catalog
    $ docker run --name demo -e 

    ...

    installCatalog=false -v ~/data/mongodb:/data/opencga/mongodb -v ~/data/solr:/data/opencga/solr -d opencb/opencga-demo:1.4.

    ...

    2

    You can connect to the OpenCGA demo container using the following command:

    Code Block
    themeRDark
    titleConnect to the OpenCGA demo container
    $ docker exec -it demo bash


    To fetch the logs of the OpenCGA demo container, use the following command: 

    Code Block
    themeRDark
    titleFetch the logs of the OpenCGA demo container
    $ docker logs demo -f


    In order to access to the Solr dashboard, use the following URL on your internet browser:

    Code Block
    titleSolr Dashboard
    http://OpenCGA-Demo-Container-IP:8983/solr


    And to access to the OpenCGA REST webservices, use the following URL:

    Code Block
    titleOpenCGA REST Server
    http://OpenCGA-Demo-Container-IP:9090/opencga/webservices/rest/v1


    How To Create OpenCGA Demo Docker Image From Source Code

    Distributed systems with multiple dependencies require a lot of work to setup, install, configure system and developers would like to have a quick way to test their changes locally and quickly with a system mimicking as close as possible to production systems. Following are the steps to create your own opencga demo image from modified oepncga source code : 


    Code Block
    themeRDark
    titleCompile OpenCGA
    ~/appl/opencga[develop*]$ mvn clean install -DskipTests -Dopencga.war.name=opencga

    Build OpenCGA Next image :

    Code Block
    themeRDark
    titleBuild OpenCGA Next Image
    ~/appl/opencga[develop*]$ docker build -t opencga-next -f opencga-app/app/cloud/docker/opencga-next/Dockerfile .

    Next Build OpenCGA Demo Image :

    Code Block
    themeRDark
    titleBuild OpenCGA Demo Image
    ~/appl/opencga[develop*]$ docker build -t opencga-demo -f opencga-app/app/cloud/docker/opencga-demo/Dockerfile .

    Catalog installation and steps to create project, study etc are in listed in "opencga-app/app/scripts/docker/opencga-demo/init.sh" script. User can modify as per own needs for example, change the variant file etc.

    Following command will run local OpenCGA Demo :

    Code Block
    themeRDark
    titleRun OpenCGA Demo Container
    ~/appl/opencga[develop*]$ docker run --name demo -d opencga-demo


    Table of Contents:

    Table of Contents
    indent20px