Date: Fri, 29 Mar 2024 12:57:23 +0000 (GMT) Message-ID: <218537887.443.1711717043304@web> Subject: Exported From Confluence MIME-Version: 1.0 Content-Type: multipart/related; boundary="----=_Part_442_1094826698.1711717043301" ------=_Part_442_1094826698.1711717043301 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Location: file:///C:/exported.html
In order to build OpenCGA from sou= rce code you must first get the source code of OpenCGA from GitHub, most of= the dependencies - including OpenCB dependencies - will be fetched from Maven Central Repository, however in some scenarios OpenCB dependencie= s will need to be built from GitHub source code. Compiling and building pro= cesses are carried out by Apache Maven. The f= ollowing tools are required for successful build:
You can learn how to install them in this section at Installation Guide > Server = Configuration.
OpenCGA as any other software has dependencies, some of them come from o= ther OpenCB projects such as CellBase while others are third-party dependen= cies such as MongoDB. All OpenCGA stable releases= are always merged and tagged at master = ;branch (users are encouraged to use latest stable release for production),= you can find all releases at OpenCGA Releases. We= guarantee that all the dependencies needed for building stable releases ar= e deployed at Maven Central Repository, this is true for b= oth OpenCB and third-party dependencies. Therefore for building a stable release you only need to clone OpenCGA r= epository itself since all the dependencies will be fetched form Maven Cent= ral repository.
This is different for development branches. Active OpenCGA development i= s carried out at develop branch, in thi= s branch third-party dependencies will be still fetched from Maven Central = Repository but this is not true for OpenCB dependenci= es since it is very likely they are still in development and therefore they= are not deployed. Keep in mind that we only guarantee that devel= op compiles and that bugs are expected, use this branch for developmen= t or for testing new functionalities. So, for building dev= elop branch you may require to download and install the = following OpenCB repositories in this order:
As you can see one of our rules is that develop branch of all m= ajor applications such as OpenCGA and CellBase always depend on develop= branches. So, if you really want to build develop the you ca= n clone and build dependencies by executing:
## Clone develop = branc git clone -b develop https://github.com/opencb/java-common-libs.git git clone -b develop https://github.com/opencb/biodata.git git clone -b develop https://github.com/opencb/cellbase.git git clone -b develop https://github.com/opencb/hpg-bigdata.git ## Now you can execute the following command in each of the folders the spe= cified order above mvn clean install -DskipTests=20
You can clone OpenCGA from GitHub by executing:
## Latest stable = version git clone -b master https://github.com/opencb/opencga.git ## Develop branch, for this to work remember to clone and build OpenCB depe= ndencies (see above) git clone -b develop https://github.com/opencb/opencga.git=20
OpenCGA allows to customise many v=
ariables in the configuration files, in order to make easier the building a=
nd configuration of OpenCGA we rely on Maven Properties that can be defined=
in file =
~/.m2/settings.xml
.=
During the building all these properties will be injected automatically in=
the configuration files so users do not have to manually change all config=
uration values. Note this is only possible when building O=
penCGA from source code, if you download the binary version you will have t=
o manually set up all configuration variables.
An example of that file can be found in the README and below. The description of e= ach property can be found below:
OPENCGA.CATALOG.DB.PASSWORD: This property should o= nly be set if the MongoDB needs authentication. In this case, this property= will contain the password of the user with permissions for the database. *= Like in the user property, this can be left empty. The admin will be able t= o set this credentials using the command line.
OPENCGA.INSTALLATION.DIR: This property is extremel= y important when using Tomcat to deploy the webservices. This property will= have to point to the final OpenCGA installation directory after everything= has been built. This property will be used by Tomcat to locate the configu= ration files. If this is not properly set, none of the webservices will wor= k. Default: /opt/opencga.
OPENCGA.USER.WORKSPACE: In Catalog, users are allow= ed to build their own directory structure, upload their own files, run anal= ysis, etc. This path should be pointing to a physical location where Catalo= g will be storing those files and directory structure. By default, we norma= lly put it in a folder called "sessions" within the installation directory = (file:///opt/opencga/sessions/).
OPENCGA.STORAGE.VARIANT.DB.HOSTS:
OPENCGA.ANALYSIS.EXECUTION.MANAGER: OpenCGA Catalog= allows users to run jobs. This property indicates how the jobs will be lau= nched. At the moment we only support two types: LOCAL to run the jobs local= ly in a thread or SGE to run the jobs using Sun Grid Engine. More queuing s= ystems will be supported soon.
OPENCGA.CLIENT.HOST: This property should be pointi= ng to the URL where the webservices will be available. For development purp= oses, the default is http://localhost:8080/opencga/. This prop= erty is read by the command line opencga.sh in order to communicate with th= e webservices.
OPENCGA.CELLBASE.REST.HOST: URL to be used for Vari= ant Annotation.
You can copy this example to ~/.m2/settings.xml:
Update
The next XML code has been updated on , please make sure you update your se= ttings.xml, sorry for any inconvenience caused.
<= ?xml version=3D"1.0" encoding=3D"UTF-8"?> <settings xmlns=3D"http://maven.apache.org/SETTINGS/1.0.0"=20 xmlns:xsi=3D"http://www.w3.org/2001/XMLSchema-instance"=20 xsi:schemaLocation=3D"http://maven.apache.org/SETTINGS/1.0.0 http://m= aven.apache.org/xsd/settings-1.0.0.xsd"> <profiles> <profile> <id>custom-config</id> <activation> <activeByDefault>true</activeByDefault> </activation> <properties> <opencga.war.name>opencga-${opencga.version}</= opencga.war.name> <!-- General --> <OPENCGA.INSTALLATION.DIR>/opt/opencga</OPENCG= A.INSTALLATION.DIR> <OPENCGA.USER.WORKSPACE>file:///opt/opencga/sessi= ons/</OPENCGA.USER.WORKSPACE> <OPENCGA.JOBS.DIR>${OPENCGA.USER.WORKSPACE}/jobs/= </OPENCGA.JOBS.DIR> <OPENCGA.DB.PREFIX>opencga</OPENCGA.DB.PREFIX&= gt; <OPENCGA.EXECUTION.MODE>LOCAL</OPENCGA.EXECUTI= ON.MODE> <!-- Client --> <OPENCGA.CLIENT.REST.HOST>http://localhost:8080/$= {opencga.war.name}</OPENCGA.CLIENT.REST.HOST> <OPENCGA.CLIENT.GRPC.HOST>http://localhost:9091&l= t;/OPENCGA.CLIENT.GRPC.HOST> <OPENCGA.CLIENT.ORGANISM.SCIENTIFIC_NAME>Homo sap= iens</OPENCGA.CLIENT.ORGANISM.SCIENTIFIC_NAME> <OPENCGA.CLIENT.ORGANISM.COMMON_NAME>human</OP= ENCGA.CLIENT.ORGANISM.COMMON_NAME> <OPENCGA.CLIENT.ORGANISM.TAXONOMY_CODE>9606</O= PENCGA.CLIENT.ORGANISM.TAXONOMY_CODE> <OPENCGA.CLIENT.ORGANISM.ASSEMBLY></OPENCGA.CL= IENT.ORGANISM.ASSEMBLY> <OPENCGA.SERVER.REST.PORT>9090</OPENCGA.SERVER= .REST.PORT> <OPENCGA.SERVER.GRPC.PORT>9091</OPENCGA.SERVER= .GRPC.PORT> <OPENCGA.MONITOR.PORT>9092</OPENCGA.MONITOR.PO= RT> <!-- Catalog --> <OPENCGA.CATALOG.DB.HOSTS>localhost:27017</OPE= NCGA.CATALOG.DB.HOSTS> <OPENCGA.CATALOG.DB.USER></OPENCGA.CATALOG.DB.= USER> <OPENCGA.CATALOG.DB.PASSWORD></OPENCGA.CATALOG= .DB.PASSWORD> <OPENCGA.CATALOG.DB.AUTHENTICATION_DATABASE></= OPENCGA.CATALOG.DB.AUTHENTICATION_DATABASE> <OPENCGA.CATALOG.DB.CONNECTIONS_PER_HOST>20</O= PENCGA.CATALOG.DB.CONNECTIONS_PER_HOST> <!-- Storage --> <OPENCGA.STORAGE.DEFAULT_ENGINE>mongodb</OPENC= GA.STORAGE.DEFAULT_ENGINE> <OPENCGA.STORAGE.CACHE.HOST>localhost:6379</OP= ENCGA.STORAGE.CACHE.HOST> <OPENCGA.STORAGE.SEARCH.HOST>http://localhost:898= 3/solr/</OPENCGA.STORAGE.SEARCH.HOST> <OPENCGA.STORAGE.STUDY_METADATA_MANAGER></OPEN= CGA.STORAGE.STUDY_METADATA_MANAGER> <!-- Storage Variants general --> <OPENCGA.STORAGE.VARIANT.DB.HOSTS>localhost:27017= </OPENCGA.STORAGE.VARIANT.DB.HOSTS> <OPENCGA.STORAGE.VARIANT.DB.USER></OPENCGA.STO= RAGE.VARIANT.DB.USER> <OPENCGA.STORAGE.VARIANT.DB.PASSWORD></OPENCGA= .STORAGE.VARIANT.DB.PASSWORD> <!-- Storage Alignments general --> <OPENCGA.STORAGE.ALIGNMENT.DB.HOSTS>localhost:270= 17</OPENCGA.STORAGE.ALIGNMENT.DB.HOSTS> <OPENCGA.STORAGE.ALIGNMENT.DB.USER></OPENCGA.S= TORAGE.ALIGNMENT.DB.USER> <OPENCGA.STORAGE.ALIGNMENT.DB.PASSWORD></OPENC= GA.STORAGE.ALIGNMENT.DB.PASSWORD> <!-- Storage-mongodb --> <OPENCGA.STORAGE.MONGODB.VARIANT.DB.AUTHENTICATION_D= ATABASE></OPENCGA.STORAGE.MONGODB.VARIANT.DB.AUTHENTICATION_DATABASE&= gt; <OPENCGA.STORAGE.MONGODB.VARIANT.DB.CONNECTIONS_PER_= HOST>20</OPENCGA.STORAGE.MONGODB.VARIANT.DB.CONNECTIONS_PER_HOST> <!-- Storage-hadoop --> <!--If empty, will use the ZOOKEEPER_QUORUM read fro= m the hbase configuration files--> <OPENCGA.STORAGE.HADOOP.VARIANT.DB.HOSTS></OPE= NCGA.STORAGE.HADOOP.VARIANT.DB.HOSTS> <OPENCGA.STORAGE.HADOOP.VARIANT.DB.USER></OPEN= CGA.STORAGE.HADOOP.VARIANT.DB.USER> <OPENCGA.STORAGE.HADOOP.VARIANT.DB.PASSWORD></= OPENCGA.STORAGE.HADOOP.VARIANT.DB.PASSWORD> <OPENCGA.STORAGE.HADOOP.VARIANT.HBASE.NAMESPACE>&= lt;/OPENCGA.STORAGE.HADOOP.VARIANT.HBASE.NAMESPACE> <OPENCGA.STORAGE.HADOOP.VARIANT.ARCHIVE.TABLE.PREFIX= >${OPENCGA.DB.PREFIX}_study</OPENCGA.STORAGE.HADOOP.VARIANT.ARCHIVE.T= ABLE.PREFIX> <!-- Email server --> <OPENCGA.MAIL.HOST></OPENCGA.MAIL.HOST> <OPENCGA.MAIL.PORT></OPENCGA.MAIL.PORT> <OPENCGA.MAIL.USER></OPENCGA.MAIL.USER> <OPENCGA.MAIL.PASSWORD></OPENCGA.MAIL.PASSWORD= > <!-- cellbase --> <OPENCGA.CELLBASE.REST.HOST>http://bioinfo.hpc.ca= m.ac.uk/cellbase/</OPENCGA.CELLBASE.REST.HOST> =09=09=09=09=09<OPENCGA.CELLBASE.VERSION>v4</OPENCGA.CELLBASE.VERS= ION> <OPENCGA.CELLBASE.DB.HOST>localhost:27017</OPE= NCGA.CELLBASE.DB.HOST> <OPENCGA.CELLBASE.DB.USER></OPENCGA.CELLBASE.D= B.USER> <OPENCGA.CELLBASE.DB.PASSWORD></OPENCGA.CELLBA= SE.DB.PASSWORD> <OPENCGA.CELLBASE.DB.AUTHENTICATION_DATABASE><= /OPENCGA.CELLBASE.DB.AUTHENTICATION_DATABASE> <OPENCGA.CELLBASE.DB.READ_PREFERENCE>secondaryPre= ferred</OPENCGA.CELLBASE.DB.READ_PREFERENCE> </properties> </profile> </profiles> </settings>=20
After creating this and configuring a default profile, you can build Ope= nCGA by executing the following command from the root of the cloned reposit= ory:
$ mvn clean insta= ll -DskipTests -P custom-config=20
The first time this command can ta= ke some minutes since it has to fetched and store locally all the dependenc= ies, next builds will be much faster. After successful building, user shoul= d find the following file structure under a build folder:
build =E2=94=9C=E2=94=80=E2=94=80 bin =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 obsolete =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 opencga-admin.sh =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 opencga-analysis.sh =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 opencga-env.sh =E2=94=82 =E2=94=94=E2=94=80=E2=94=80 opencga.sh =E2=94=9C=E2=94=80=E2=94=80 conf =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 client-configuration.yml =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 configuration.yml =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 log4j.properties =E2=94=82 =E2=94=94=E2=94=80=E2=94=80 storage-configuration.yml =E2=94=9C=E2=94=80=E2=94=80 examples =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 1k.chr1.phase3_shapeit2_mvncall_int= egrated_v5.20130502.genotypes.vcf.gz =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 20130606_g1k.ped =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 opencga-index.sh =E2=94=82 =E2=94=94=E2=94=80=E2=94=80 opencga-storage-fetch.sh =E2=94=9C=E2=94=80=E2=94=80 libs =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 activation-1.1.jar =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 ..... =E2=94=82 =E2=94=94=E2=94=80=E2=94=80 zookeeper-3.4.6.jar =E2=94=9C=E2=94=80=E2=94=80 LICENSE =E2=94=9C=E2=94=80=E2=94=80 opencga-1.0.0-rc4.war =E2=94=9C=E2=94=80=E2=94=80 README.md =E2=94=9C=E2=94=80=E2=94=80 test =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 bin =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 dependencies =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 fitnesse =E2=94=82 =E2=94=94=E2=94=80=E2=94=80 README.md =E2=94=94=E2=94=80=E2=94=80 tools =E2=94=9C=E2=94=80=E2=94=80 affy-expression-normalization =E2=94=9C=E2=94=80=E2=94=80 ..... =E2=94=94=E2=94=80=E2=94=80 variant 42 directories, 181 files=20
Table of Contents: