Instructions for Admin

Installation on Ubuntu 18.04. LTS

Run the following instructions as root (this means, type sudo in front of every command) in order to have the root rights. If a file needs to be opened/modified, use the editor “nano” (type nano in front of the filename).

1. Ubuntu packages

   $ apt install git r-base postgresql python3-pip python3-psycopg2
   

2. Install julia 1.0.1 and create a symbolic link: (Ubuntu offers julia 0.4.x only):

   $ wget https://julialang-s3.julialang.org/bin/linux/x64/1.0/julia-1.0.1-linux-x86_64.tar.gz
   $ tar xzf julia-1.0.1-linux-x86_64.tar.gz
   $ mv julia-1.0.1 /opt
   $ ln -s /opt/julia-1.0.1/bin/julia /usr/local/bin/
   $ julia --version
   

3. Create data base users and database

We create a user whdp who owns the whdp database and thus has all permissions to modify tables and data in this database. In addition to that we create a whdp_reader user which only has read access.

$ sudo -u postgres createuser -P whdp
$ sudo -u postgres createuser -P whdp_reader

Note down the choosen passwords for both users.

Then create the database and restrict the permissions of whdp_reader.

$ sudo -u postgres createdb -O whdp whdp
$ sudo -u postgres psql -c "REVOKE ALL PRIVILEGES ON ALL TABLES IN SCHEMA public FROM whdp_reader;"
$ sudo -u postgres psql -c "GRANT SELECT ON ALL TABLES IN SCHEMA public TO whdp_reader;"

Now check:

$ psql -U whdp -h 127.0.0.1 whdp
$ ^D    (control-D to exit postgress shell)

Next, the database must be configured to allow for remote access.

In the file /etc/postgresql/10/main/pg_hba.conf edit the line host    all             all             127.0.0.1/32              md5 to host    all             all             0.0.0.0/0                 md5 .

In the file /etc/postgresql/10/main/postgresql.conf change # listen_addresses = 'localhost' to listen_addresses = '*'.

Restart the data base:

$ service postgresql restart

4. Install WHDP

   $ cd /opt
   $ git clone https://sissource.ethz.ch/sispub/whdp.git
   $ apt install python3-pip
   $ pip3 install -e whdp
   

   Check installation:

   $ whdp --help
   

   Install needed packages for demo scripts:

   $ /opt/whdp/scripts/setup_julia_et_al.sh
   

5. Create user account for data provider:

   $ addgroup whdp
   $ useradd -m -G whdp,systemd-journal -s /bin/bash whdp-provider
   

   Assign password:

   $ passwd whdp-provider
   

6. Initialize WHDP configuration and setup landing zone:

   We assume that the landing zone will be located on a shared drive mounted at /nfsmount, but you are free to choose any other folder.

   Create the landing zone and link it to the WHDP:

   $ mkdir -p /nfsmount/landing_zone
   $ whdp init-config /nfsmount/landing_zone
   

   Set the correct permissions:

   $ chgrp -R whdp /nfsmount/landing_zone
   $ chmod -R g+w /nfsmount/landing_zone
   

7. Adapt configuration:

   $ /etc/whdp/whdp.ini
   

   Add the database user and password. Replace DB_USER and DB_PASSWORD with the one selected in step 3.

   ...
   [db]
   connection_string = postgresql://DB_USER:DB_PASSWORD@127.0.0.1:5432/whdp
   

   If necessary adapt also the path to the landing zone, define a backup landingzone, or change software versions.

   Then check:

   $ whdp check-config
   

8. Create the central management tool service for controlling the init system:

$ ln -s /opt/whdp/scripts/whdp.service /etc/systemd/system
$ systemctl daemon-reload

9. Start service:

$ systemctl start whdp.service
$ systemctl status whdp.service

10. Observe running service:

can be stopped with ^C), can be used without -f:

$ journalctl -u whdp -f

Keep this terminal window open if you want observe the whdp activities.

11. Install julia packages:

Login as user whdp-provider first.

Install needed Julia packages (these are installed per user) to be able to run the test scripts:

$ /opt/whdp/scripts/setup_julia.sh