Page History

...

By default, the variant annotator will call remote web services deployed at the University of Cambridge (http://bioinfo.hpc.cam.ac.uk/cellbase/webservices/). If a local installation of the CellBase database is available, the --local flag can be enabled to improve annotation performance; please, check the github readme (https://github.com/opencb/cellbase) to learn how to build the code and point it to a particular local database.

Variant annotation including custom files

In many occassions, variant annotation users want to complement core CellBase annotation with custom annotations either from own generated files or other resources not currently integrated in CellBase database. Thus, the variant annotation command line allows the user can specify a set of files with custom annotation data. Currently, only VCF files are allowed as custom input. Any attribute value in the INFO field can be used during the annotation process.

Three parameters in the command line control custom annotation behaviour:

          --custom-file                STRING     String with a comma separated list (no spaces in between) of files with custom annotation to be
                                                  included during the annotation process. File format must be VCF. For example: file1.vcf,file2.vcf
          --custom-file-fields         STRING     String containing a colon separated list (no spaces in between) of field lists which indicate the
                                                  info fields to be taken from each VCF file. For example:
                                                  field1File1,field2File1:field1File2,field3File2
          --custom-file-id             STRING     String with a comma separated list (no spaces in between) of short identifiers for each custom file.
                                                  For example: fileId1,fileId2

1.  First, you will need to specify the set of files that should be used as a custom annotation input. You can do this by using the `--custom-file` parameter and providing a comma separated list (no spaces in between) of VCF files. For example, /tmp/file1.vcf,/tmp/file2.vcf,/tmp/file3.vcf.
2. Next, you need to specify which INFO attributes must be taken from each of the files. You can do this by using the `--custom-file-fields` parameter, and providing a colon separated list (no spaces in between) of info tag lists. For example, field1File1,field2File1:field1File2,field3File2
3. Last, you need to provide one label or tag per file. This label will be used in the output file to represent this file in the variant annotation ouput. You can do this by using the `custom-file-id` parameter, and provinding a comma separated list (no spaces in between) of short labels/tags/identifiers.

A variant-annotation command including three custom files will look like:

$ cellbase$ build/bin/cellbase.sh variant-annotation -i /tmp/test.vcf.gz \
  -o /tmp/test.json.gz \
  --species hsapiens \
  --assembly GRCh37 \
  --custom-file /tmp/file1.vcf,/tmp/file2.vcf,/tmp/file3.vcf \
  --custom-file-fields AF,DP:AF,DP:AF,DP
  --custom-file-id file1,file2,file3

In this example, three files (file1.vcf, file2.vcf, file3.vcf) will be used as a source of custom annotation. Values of the AF and DP attributes will be the annotation data taken from the three files. Finally, labels `file1`, `file2`, `file3` will identify the file from which each AF and DP value were read from.

If annotating with custom files, the first step run by the `variant-annotation` command will consist of creating a RocksDB database per custom file. RocksDB (rocksdb.org) is a high performance key-value store which has shown to be extremely useful for this kind of use-case. Thus, each database will contain one entry for each variant in the VCF; the key being a variant id and the value the corresponding AF value. The RocksDB index path will share the prefix with the custom file and will end in `.idx`. In our example, three RocksDB indexes would be created:

/tmp/file1.vcf.idx
/tmp/file2.vcf.idx
/tmp/file3.vcf.idx

It is important to note that these indexes need to be created just once. Posterior runs of the variant-annotation command using any of these custom files wont need to re-index them and will directly read from the index. Please, also note, that if the INFO attributes to be used vary, these indexes must be manually removed from the filesystem to force the variant-annotation tool to re-index the three files with the new attributes.

During the annotation process, the annotator will merge core annotation coming from the Mongo queries to the main CellBase database, with these custom annotations coming from queries to the RocksDB indexes. Both annotation results will be integrated into VariantAnnotation objects (https://github.com/opencb/biodata/blob/develop/biodata-models/src/main/avro/variantAnnotation.avdl). Custom annotations will be stored within the additionalAttributes field, looking like this:

{
  "names": [],
  "chromosome": "1",
  "start": 10428,
  "reference": "",
  "alternate": "C",
  "studies": [ ... ],
  "type": "INDEL",
  "annotation": {
    "chromosome": "1",
    "start": 10428,
    "reference": "",
    "alternate": "C",
    "displayConsequenceType": "2KB_upstream_variant",
    "consequenceTypes": [ ... ],
    "additionalAttributes": {
      "file1": {
        "attribute": {
          "AF": "0.82",
          "DP": "4"
        }
      },
      "file2": {
        "attribute": {
          "AF": "0.32",
          "DP": "42"
        }
      },
      "file3": {
        "attribute": {
          "AF": "0.2",
          "DP": "14"
        }
      }
    }
  },
  "end": 10427,
  "strand": "+",
  "hgvs": {},
  "length": 1
}

Using the Python client (PyCellBase)

...