PayControl Inform. Installation guide

This is PayControl Inform Installation guide. Main article about PayControl Inform and it's API — PayControl Inform.

Installation

Install pre-requisits

EPEL

Add EPEL repository and install another necessary packages (git needed, only if server has ability to connect to git repository with pushgate sources):

sudo yum install epel-release
sudo yum install wget libffi-devel openssl-devel gcc git

Erlang

Install Erlang (needed for RabbitMQ).

Centos 7

wget https://packages.erlang-solutions.com/erlang-solutions-1.0-1.noarch.rpm
sudo rpm -Uvh erlang-solutions-1.0-1.noarch.rpm
sudo yum install erlang

Centos 8

sudo dnf install erlang

RabbitMQ

Install RabbitMQ, enable and start it.

curl -s https://packagecloud.io/install/repositories/rabbitmq/rabbitmq-server/script.rpm.sh | sudo bash
sudo yum install rabbitmq-server
sudo systemctl start rabbitmq-server
sudo systemctl enable rabbitmq-server

Set up user-login for RabbitMQ. Enable Managemet plugin (it wil accessible on HTTP:15672/TCP). Grant access through firewall from Administrator's workstation to management console.

rabbitmqctl add_user <USERNAME> <PASSWORD>
rabbitmqctl set_user_tags <USERNAME> administrator
rabbitmqctl set_permissions -p / <USERNAME> ".*" ".*" ".*"
rabbitmq-plugins enable rabbitmq_management
firewall-cmd --get-active-zones
firewall-cmd --permanent --zone=<ZONE RETURNED BY PREVIOUS COMMAND> --add-rich-rule='
  rule family="ipv4"
  source address="<SOURCE IP/MASK>"
  port protocol="tcp" port="15672" accept'
firewall-cmd --reload

Redis

Install, start and enable Redis (apt-get install redis-server)

sudo yum install redis
sudo systemctl start redis
sudo systemctl enable redis

PostgreSQL

Install PostgreSQL and set up user-login, create database, do not forget to allow remote tcp-connection for user

sudo yum install postgresql postgresql-server
sudo postgresql-setup initdb
sudo systemctl start postgresql
sudo systemctl enable postgresql
sudo vi /var/lib/pgsql/data/postgresql.conf
    listen_addresses = 'localhost' ## Uncomment this string in /var/lib/pgsql/data/postgresql.conf
sudo vi /var/lib/pgsql/data/pg_hba.conf
    host    all             all             127.0.0.1/32            md5 ## Change ident to md5 in /var/lib/pgsql/data/pg_hba.conf
    host    all             all             ::1/128                 md5 ## Change ident to md5 in /var/lib/pgsql/data/pg_hba.conf
sudo -u postgres psql
create database <DB_NAME>;
create user <DB_USER> password '<DB_USER_S_PASSWORD>';
grant all privileges on database <DB_NAME> to <DB_USER>;
\q
psql -h localhost -p 5432 -U <DB_USER> -d <DB_NAME> -W

Insert and execute SQL script:

create table tasks (
	id serial,
	uuid uuid not null unique,
	device_token text,
	payload text,
	callback_url text,
	created_at timestamp without time zone not null default (now() at time zone 'utc'),
	updated_at timestamp without time zone not null default (now() at time zone 'utc')
);


create table task_statuses (
	id serial,
	uuid uuid references tasks (uuid) on delete cascade,
	status smallint not null default 0,
	dt timestamp without time zone not null default (now() at time zone 'utc')
);


create table housekeeping_stat (
	total bigint
);

insert into housekeeping_stat (total) values (0);

create index ON tasks(created_at);

create index ON task_statuses(uuid);

Periodically you must reindex DB:

REINDEX DATABASE <DB_NAME>;

Owner of new tables must be <DB_USER>. Else, while housekeeper starts, may occours the error: asyncpg.exceptions.InsufficientPrivilegeError: permission denied for relation tasks. In this case you must set ownership of all tables in <DB NAME> to <DB_USER>:

ALTER TABLE task_statuses OWNER TO <DB_USER>;
ALTER TABLE tasks OWNER TO <DB_USER>;
ALTER TABLE housekeeping_stat OWNER TO <DB_USER>;

Phython

Centos 7

For Centos 7 you must install pithon 3.6 from IUS repository (in packages's names added letter "u"):

sudo yum install https://centos7.iuscommunity.org/ius-release.rpm
sudo yum install python36u python36u-devel python36u-setuptools

Instal virtualenv:

sudo pip3 install virtualenv

Centos 8

sudo dnf install python36 python36u-devel

Install virtualenv:

sudo pip3 install virtualenv

Create vitual environment once for all pushgate services (use python 3.6 only)

mkdir -p /var/app/
virtualenv -p python3.6 /var/app/.env

Install services

cd /var/app
source .env/bin/activate

Next step is putting files into current directory. There are two ways:

• Extract archives recieved from SafeTech.
• From Safe-Tech Repository (throught internal network only). Example:

  git clone git@gitlab.paycontrol.org:pushgate/pushgate.git
  git clone git@gitlab.paycontrol.org:pushgate/tasker.git
  git clone git@gitlab.paycontrol.org:pushgate/push-secretary.git
  git clone git@gitlab.paycontrol.org:pushgate/payloader.git
  git clone git@gitlab.paycontrol.org:pushgate/apple-pusher.git
  git clone git@gitlab.paycontrol.org:pushgate/google-pusher.git
  git clone git@gitlab.paycontrol.org:pushgate/callbacker.git
  git clone git@gitlab.paycontrol.org:pushgate/status-porter.git
  git clone git@gitlab.paycontrol.org:pushgate/housekeeper.git
  

When PCInform files is on the place, proceed installation:

pip install -e pushgate
pip install -e tasker
pip install -e push-secretary
pip install -e payloader
pip install -e apple-pusher
pip install -e google-pusher
pip install -e callbacker
pip install -e status-porter
pip install -e housekeeper

Fix of version mismatches

Because of using multiple libraries may occurs some version mismatches between of them. For example:

Errors may occurs, during google-pusher installation:

yarl 1.2.4 has requirement multidict>=4.0, but you'll have multidict 2.1.6 which is incompatible.
aiohttp 3.3.1 has requirement multidict<5.0,>=4.0, but you'll have multidict 2.1.6 which is incompatible.

Errors may occurs, during callbacker, status-porter and housekeeper installation:

aioxmpp 0.9.1 has requirement multidict~=2.0, but you'll have multidict 4.3.1 which is incompatible.

In this case, you need to:

pip uninstall multidict
pip uninstall yarl
pip uninstall aiohttp
pip install multidict==2.1.6
pip install aiohttp==2.1.0

Configure services

cd /var/app
git clone git@gitlab.paycontrol.org:pushgate/configuration.git
cp configuration/pushgate.conf.DEFAULT configuration/pushgate.conf
vi configuration/pushgate.conf

Add your APNS and Firebase credentials to pushgate.conf.

cp -R configuration/services/* /etc/systemd/system

Config DB connection in /var/app/configuration/pushgate.conf

DB_DSN=postgres://<DB_USER>:<DB_USER_S_PASSWORD>@127.0.0.1/<DB_NAME>

Configure SELinux

For Centos 8 you need to configure SELinux. At first, install SELinux utils:

dnf install policycoreutils-python-utils

Then set contexts:

semanage fcontext -a -t var_run_t  /var/app/configuration/pushgate.conf
semanage fcontext -a -t bin_t /var/app/.env/bin/apple-pusher
semanage fcontext -a -t bin_t /var/app/.env/bin/callbacker
semanage fcontext -a -t bin_t /var/app/.env/bin/google-pusher
semanage fcontext -a -t bin_t /var/app/.env/bin/housekeeper
semanage fcontext -a -t bin_t /var/app/.env/bin/payloader
semanage fcontext -a -t bin_t /var/app/.env/bin/push-secretary
semanage fcontext -a -t bin_t /var/app/.env/bin/status-porter
restorecon -v /var/app/configuration/pushgate.conf
restorecon -Rv /var/app/.env/bin/

Start services

systemctl start push_secretary
systemctl start payloader
systemctl start apple_pusher@{1..6}
systemctl start google_pusher@{1..8}
systemctl start callbacker@{1..2}
systemctl start status_porter@{1..2}
systemctl start housekeeper

Check services status

systemctl status push_secretary
systemctl status push_secretary | grep '.service - \|Active'
systemctl status payloader | grep '.service - \|Active'
systemctl status apple_pusher@{1..6} | grep '.service - \|Active'
systemctl status google_pusher@{1..8} | grep '.service - \|Active'
systemctl status callbacker@{1..2} | grep '.service - \|Active'
systemctl status status_porter@{1..2} | grep '.service - \|Active'
systemctl status housekeeper | grep '.service - \|Active'

While housekeeper had not started with error:
"ConnectionRefusedError: [Errno 111] Connection refused",
other services had started, but in status of services contains errors, like:
"ERROR amqp:073 Couldn't connect to RabbitMQ server to consume ([Errno 111] Connection refused)"
add to /etc/systemd/system/*.service file, in [Unit] section, (where * is apple_pusher@, google_pusher@, callbacker, status_porter, housekeeper):

After=rabbitmq-server.service

While after request to localhost:80/8080 payloader/push_secretary get errors, you need to change current version uvloop to 0.8.1

pip uninstall uvloop
pip install uvloop==0.8.1

Enable services

systemctl enable push_secretary
systemctl enable payloader
systemctl enable apple_pusher@{1..6}
systemctl enable google_pusher@{1..8}
systemctl enable callbacker@{1..2}
systemctl enable status_porter@{1..2}
systemctl enable housekeeper

At this step you may set up number of processes. For example, if you need just one of apple_pusher process, you need to correct systemctl enable command like this:

systemctl enable apple_pusher@1

Ajusting quantity of push_secretary and payloader serveces is in the configuration/pushgate.conf file, described here.

Reinstall (update) services

Backup

Backup all files, that was modified, or you want to save during update. In example it may be:

• configuration file configuration/pushgate.conf;
• APNS certificates.

Stop services

systemctl stop push_secretary
systemctl stop payloader
systemctl stop apple_pusher@{1..6}
systemctl stop google_pusher@{1..8}
systemctl stop callbacker@{1..2}
systemctl stop status_porter@{1..2}
systemctl stop housekeeper

Change environment

cd /var/app
source .env/bin/activate

Update files

Next step is putting new files into current directory. There are two ways:

• Extract archives recieved from SafeTech.
• From Safe-Tech Repository (throught internal network only). Example:

cd apple-pusher
git checkout .
git pull
cd ../callbacker
git checkout .
git pull
cd ../configuration
git checkout .
git pull
cd ../google-pusher
git checkout .
git pull
cd ../housekeeper
git checkout .
git pull
cd ../payloader
git checkout .
git pull
cd ../pushgate
git checkout .
git pull
cd ../push-secretary
git checkout .
git pull
cd ../status-porter
git checkout .
git pull
cd ../tasker
git checkout .
git pull
cd ../

Update components

Update pip:

pip install --upgrade pip

Update PCInform components:

pip install --upgrade -e pushgate
pip install --upgrade -e tasker
pip install --upgrade -e push-secretary
pip install --upgrade -e payloader
pip install --upgrade -e apple-pusher
pip install --upgrade -e google-pusher
pip install --upgrade -e callbacker
pip install --upgrade -e status-porter
pip install --upgrade -e housekeeper

Start services

systemctl start push_secretary
sleep 3
systemctl status push_secretary
sleep 1
systemctl start payloader
sleep 3
systemctl status payloader
sleep 1
systemctl start apple_pusher@{1..6}
sleep 3
systemctl status apple_pusher@{1..6}
sleep 1
systemctl start google_pusher@{1..8}
sleep 3
systemctl status google_pusher@{1..8}
sleep 1
systemctl start callbacker@{1..2}
sleep 3
systemctl status callbacker@{1..2}
sleep 1
systemctl start status_porter@{1..2}
sleep 3
systemctl status status_porter@{1..2}
sleep 1
systemctl start housekeeper
sleep 3
systemctl status housekeeper

Customization

Optimum services count for 20 CPU

push_secretary workers count = 3
payloader workers count = 3
apple_pusher@{1..6}
google_pusher@{1..8}
status_porter@{1..2}
callbacker@{1..2}```

Set up NGINX for SSL/TLS-Offload for payloader

Use Google, something like this https://www.digitalocean.com/community/tutorials/how-to-set-up-nginx-load-balancing-with-ssl-termination