Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Overview

Understanding the URL

The general format of the REST API web services is: 

Code Block
themeEclipse
titleOpenCGA RESTful URL Structure
http://HOST_URL/webservices/rest/{apiVersion}/{entity}/{id(s)}/{endpoint}?{options}

where HOST_URL is the name of the host machine and the name of Java war file deployed in web server (eg. tomcat) over that server for example bioinfodev.hpc.cam.ac.uk/opencga

Entities inside the curly braces { } are the web service parameters, and they are treated as variables. For example the following URL:

Code Block
themeEclipse
titleQuick Example
http://bioinfodev.hpc.cam.ac.uk/opencga/webservices/rest/v1/users/opencga/login?password=opencga

As is explained later in this documentation, this RESTful web service will login the user opencga.

  • apiVersion (v1) : indicates OpenCGA version to retrieve information from, data models and API may change between versions. 
  • entity (users) : specifies the data type of what the user wants to retrieve in the id field. This can be one of resources listed below. 
  • id (opencga) : the resource id we want to query
  • endpoint (login) : these parameters must be specified depending on the nature of your input data. For example, if we want to query all files by a specific study (e.g. 1000genomes) we should use the studies resource and files endpoint. 
  • options (password) : variables in key value pair form, passed as query parameters.

URL parameters

apiVersion

apiVersions are numbered as v1v2, etc. At this moment we are heading to second stable apiVersion which will be v2.

entity

There are several metadata entities implemented such as users, samples, individuals, ... see below for more info..

IDs

This is the query parameter and the type matches resources path parameter, this is a unique identifier for the resource we are interacting. Plural means a list of IDs can be passed to improve performance with a single REST call separated by commas rather than multiple calls. OpenCGA preserves the order of the results with corresponding IDs. A Boolean variable, silent, can be set to indicate either if user is interested to receive partial results (true) as well or only expects complete set of results or nothing in case of any failure (resource doesn't exist, permission denied etc). As a trade off between performance and ease of use a maximum of 100 IDs are allowed in one web service.

options

These query parameters can modify the behaviour of the query (excludeincludelimitskip and count) or add some filters to some specific endpoints to add useful functionality. The following image shows some typical options for a certain web service. 

REST Response

Most web services return the results encapsulated in a single QueryResponse object (view data model) consisting of some metadata and a list of QueryResult objects (view data model) called response containing the data and metadata requested. The reason for this two-level response is that some REST web services allow to pass multiple IDs as input parameter, this improves significantly the performance by reducing the number of calls, for instance a calling /info method with three sample IDs will return a QueryResponse object with three QueryResults. Then, each QueryResult can contain multiple results, for instance when getting all samples from an individual or when fetching all variants from a gene.

However, most of the web services will return a QueryResponse with one single QueryResult with one or more result. In general the response object looks like:

Code Block
titleQueryResponse Object
linenumberstrue
{
  "apiVersion": "v1",
  "time": 19,
  "warning": "",
  "error": "",
  "queryOptions": {
    "metadata": true,
    "skipCount": false,
    "limit": 10
  },
  "response": [
    {
      "id": "search",
      "dbTime": 18,
      "numResults": 10,
      "numTotalResults": 56,
      "warningMsg": "",
      "errorMsg": "",
      "resultType": "",
      "result": [
        {
          	// result 1
        },
		{
          	// result 2
        },
		// ...
		{
			// result 10
        }
      ]
    }
  ]
}


where:

  • Line 1: single QueryResponse object
  • Lines 2 and 3: show the version and the duration time (ms)
  • Lines 4 and 5: show warning and error messages, for instance when having network issues you could get "Catalog database not accessible"
  • Line 6: summary of all option parameters provided
  • Line 11: list of QueryResults called response. In this example, and in most of calls, there is only one QueryResult.
  • Line 14: database duration time (ms) for each QueryResult.
  • Line 15 and 16: number of elements returned in the list result (see below) and total number of records found in the database for a given query.
  • Line 17 and 18: specific warning and error messages for each QueryResult
  • Line 19: type of result such as entity
  • Line 20: list of results for this query, this can be samples, variants, ...

Entities and Endpoints

REST API is organised in two main groups of web services depending you want to work with metadata and permission or run some analysis: Catalog and Analysis. See below a description of the web services.

Catalog Web Services

Contains all endpoints for managing and querying metadata and permission.

EntityPathDescriptionMain Endpoints
Users/usersDifferent methods to work with usersinfo, create, login, ...
Projects/projectsprojects are defined for each user and contains studiesinfo, create, studies, ...
Studies/studiesstudies are the main component of catalog, the can be shared and contain files, samples and jobsinfo, create, files, samples, jobs, variants, alignments, groups, ...
Files/filesfiles are added to the study and can be indexed to be queriedinfo, create, index, share, ...
Jobs/jobsjobs are tool executions that can be queuedinfo, create, ...
Families/familiesfamilies are connected collection of individuals based on relationshipinfo, create, ...
Individuals/individualssamples come from the individualsinfo, create, ...
Samples/samplessamples are each of the experiment samples, typically matches a NGS BAM file or VCF sampleinfo, create, annotate, share, ...
Cohorts/cohortsthese model a group of samples that share some common properties, these are used for data analysisinfo, create, stats, samples, ...
Clinical Analysis/clinicalthis handles creating and search of a clinical analysisinfo, create, ...
Variable Set/variablesetvariables annotate samples with different information useful for data analysisinfo, crate, ...
Meta/metagives the meta information about OpenCGA installation instanceping, about, status
GA4GH/ga4ghGA4GH standard web services to search genomics data in OpenCGAvariant search, reads search, responses

Analysis Web Services

Different endpoint for running the alignment, variant and clinical analysis

CategoryPathDescriptionMain Endpoints
Alignment Analysis/analysis/alignmentOperations over Read Alignments to facilitate complete analysis with different tools.index, query, stats, coverage
Variant Analysis/analysis/variant

Operations over Genomic Variants to facilitate complete analysis with different tools.

index, stats, query, validate, ibs, facet, samples, metadata
Clinical Analysis/analysis/clinicalYou can manage Clinical Analysis metadata (e.g create a case, set permissions) or run a genome interpretationexecute

Swagger

OpenCGA has been documented using Swagger project. Detailed information about resourcesendpoints and options is available at:

http://bioinfodev.hpc.cam.ac.uk/opencga/webservices/

Client Libraries

Currently OpenCGA implements the following four client libraries: 

  1. Java
  2. Python
  3. R
  4. JavaScript

Deprecation Policy

Certain APIs are deprecated over the period of time as OpenCGA is a live project and continuously improved and new features are implemented. Deprecation cycle consists of a warning period to let make user aware that these services are considered for change and highly likely will be replaced followed by a deprecated message. OpenCGA supports deprecated services for two releases (Deprecated and Next one). Deprecated services are hidden in the following release of deprecation and finally removed completely in next release.

Code Block
themeEclipse
titleDeprecation Life Cycle
Warning (working) ---> Deprecated (working) ---> Hidden (working) ---> Removed (not working) 

All deprecated web services are clearly marked as deprecated along with a warning help message for user.


Table of Contents:

Table of Contents
indent20px