Cloning a B-Translator Server

First, I build a new server from scratch:

mkdir /var/chroot
cd /var/chroot/
git clone --branch v1.0 https://github.com/dashohoxha/B-Translator.git
nohup nice B-Translator/install/install.sh btr-1 &
tail -f nohup.out

Then configure it:

chroot btr-1/ /tmp/install/config.sh

And then reboot the host:

reboot  ## it is advisable to reboot the host after this installation

The last step of configuration is about setting the primary language of translation (sq in my case), and a list of auxiliary languages. This list should include at least fr, since it is used instead of POT (PO template) files, when they are missing. But the features related to auxiliary languages are not implemented yet, so it is not much useful to include other languages on that list, and I leave just fr.

If I can assign a domain name to this copy of B-Translator (for example test.l10n.fs.al), but during the configuration it is not given properly, then I can correct/change it like this:

chroot /var/chroot/btr-1
cd /var/www/btr/profiles/btranslator
install/config/domain.sh

If this is a local copy (installed on my local machine) and its domain name is for example l10n.org.xy, then I also add this domain name on the file /etc/hosts of my machine:

127.0.0.1       l10n.org.xy  localhost

This way I can access it by typing https://l10n.org.xy on the browser location (not https://127.0.0.1 or https://localhost).

I have used NGINX as the default web server because it is supposed to be faster and more responsive under heavy load. Unfortunately it has been working very slow, for some reasons that I have not been able to find and fix. On the log file /var/log/nginx/btr.error.log I see messages like this:

2013/07/15 13:31:57 [info] 10294#0: *47 client closed prematurely connection while SSL handshaking, client: 127.0.0.1, server: l10n.org.xx
2013/07/15 13:31:57 [info] 10294#0: *48 client closed prematurely connection while SSL handshaking, client: 127.0.0.1, server: l10n.org.xx

Note: If find out what is wrong with NGINX and manage to fix it, please let me know.

So, I replace NGINX with Apache2 like this:

chroot /var/chroot/btr-1
cd /var/www/btr/profiles/btranslator
dev/apache2.sh start

It will stop services nginx, php5-fpm and memcached, will start apache2, will disable drupal module memcache, and will modify properly settings.php.

However there are also some changes outside chroot that should be done. The script dev/apache2.sh start cannot do them automatically (because it runs inside the chroot), so I have to do them manually. On the init script /etc/init.d/chroot-btr-1 I make this change:

#CHROOT_SERVICES="cron php5-fpm memcached mysql nginx"
CHROOT_SERVICES="cron mysql apache2"

After installation I also enable some features that are optional and are not enabled by default during installation:

drush features-list
drush --yes pm-enable btr_simplenews
drush --yes pm-enable btr_mass_contact
## drush --yes pm-enable btr_googleanalytics
## drush --yes pm-enable btr_drupalchat

If this copy of B-Translator is remote, then I install ssh as well for accessing it easily and for using remote drush commands:

chroot /var/chroot/btr-1
cd /var/www/btr/profiles/btranslator
dev/install-sshd.sh

This script will also take care to change the ssh port to 2201, in order to avoid any conflicts with any existing daemon on the host environment, and also for increased security.

For drush remote access to work correctly, the public/private key ssh access should be set up and configured as well. For more detailed instructions on how to do it see: http://dashohoxha.blogspot.com/2012/08/how-to-secure-ubuntu-server.html

I almost always forget the password of admin (the primary user of Drupal) that I assign during installation. So, I have to reset it:

drush user-password admin --password="new-password"

By the way, on the file /etc/drush/drushrc.php you can see this drush setting:

<?php
// by default use the B-Translator root directory
$options['r'] = '/var/www/btr';

This means that the root directory of drush is always /var/www/btr, no matter where we call it from.

On the master (main/live) server, I export all the content as a feature, with the help of the module node_export. But first I have to disable the existing btr_content feature, otherwise the feature export will fail.

drush --yes pm-disable btr_content
drush --yes features-export --destination=/tmp btr_content_1 node_export_features
tar --create --gzip --file=btr_content_1.tgz --directory=/tmp btr_content_1

Now I transfer btr_content_1.tgz to the clone server and replace the existing content with it:

cd /var/www/btr/profiles/btranslator
cd modules/features/
tar --extract --gunzip --file=btr_content_1.tgz
drush --yes pm-disable btr_content
drush delete-all all  ## delete all existing nodes
drush --yes pm-enable btr_content_1

Path aliases and some menus have to be fixed (recreated) manually. I couldn't find any modules, drush commands or scripts that can transfer them automatically. If you know any tricks to export/import them automatically, please let me know.

I transfer manually the configuration of the Homebox as well. I open admin/structure/homebox on both sites (the main and the clone), export the configuration of 'dashboard' from the main, then copy/paste and import it on the clone.

Private settings are those variables that are site specific and cannot be included in features, for example: disqus_domain, disqus_userapikey, disqus_publickey, disqus_secretkey, etc.

We can transfer them like this:

• Save them on the main site:

  cd /var/www/btr/profiles/btranslator
  modules/features/save-private-vars.sh
  

  It will generate the file restore-private-vars.php.

• Transfer restore-private-vars.php to the clone site and then apply it like this:

  drush php-script restore-private-vars.php
  

Note: If the clone site will be used for testing, you may consider to edit manually the file restore-private-vars.php, before applying it, and change some values. For example, I usually change email addresses from info@l10n.fs.al to info+test@l10n.fs.al. I also enable email rerouting by changing these variables:

$variables['reroute_email_enable'] = 1;
$variables['reroute_email_enable_message'] = 1;

Now the cloned site is almost identical with the primary site in terms of Drupal settings and configuration and in terms of translation data (projects that are imported, strings and their translations, etc.). What is still missing is the list of users that are registered on the primary site, as well as their contributions (votes and suggested translations).

We can transfer them like this:

• Export them on the primary site:

  cd /var/www/data
  db/export-users.sh
  db/export-contributions.sh
  

• Transfer *.sql.gz files to the clone site and import them:

  cd /var/www/data
  db/import-users.sh  users-20130717.sql.gz
  db/import-contributions.sh contributions-00000000-20130717.sql.gz
  

Note: Once you have transferred all the users, the clone site will send them daily a string for review, in addition to the one that is sent by the primary site. We can prevent this by disabling the cron. Edit the file /etc/cron.d/drupal7 and comment the line that starts the cron.

Vocabulary is a pseudo-project, its PO file does not really belong to the translation of any program. I use it to collect interesting terms and translations that I encounter while translating the other projects. It can help as a reminder (in case that I forget the translation of a term). It is also useful for discussing translations of difficult terms with other people, and indirectly it helps to ensure consistency among the translations. Terms are added to vocabulary by the translators. In order to transfer them from one instance of B-Translator to another, it can be exported as a PO file on one system and imported to the other.

• Export vocabulary-sq.po:

  cd /var/www/data
  export/export.sh misc vocabulary sq $(pwd)/tmp
  mv tmp/vocabulary/vocabulary-sq.po .
  rm -rf tmp
  

• Transfer vocabulary-sq.po to the other system and them import it:

  cd /var/www/data
  mv vocabulary-sq.po vocabulary/
  import/vocabulary.sh
  

The database of translations is almost empty (it has only the PO files that were imported for testing during installation). Downloading and importing all the PO files is easy (but it takes a long time).

Suppose that I want to make the cloned server primary. In this case there are some steps that should be done:

• Transfer GoogleApps verification files. I use GoogleApps for email accounts etc. (it offers 10 email accounts for free). To verify that I own this domain, GoogleApps requests me to put a certain file on the root of my webserver. This file looks like google9350a51ac2d503bf.html and I place it on /var/www/btr.
• Transfer SSL certificates. I have obtained a free SSL certificate for my site (see: http://arstechnica.com/security/2009/12/how-to-get-set-with-a-secure-sertificate-for-free/). The configuration on /etc/nginx/sites-available/default looks like this:

  ssl_certificate         /etc/ssl/certs/ssl-cert-l10n_fs_al.pem;
  ssl_certificate_key     /etc/ssl/private/ssl-cert-l10n_fs_al.key;
  

  The corresponding configuration on /etc/apache2/sites-available/default-ssl looks like this:

  SSLCertificateFile    /etc/ssl/certs/ssl-cert-l10n_fs_al.pem
  SSLCertificateKeyFile /etc/ssl/private/ssl-cert-l10n_fs_al.key
  

  The files .pem and .key need to be transferred to the new server and the configuration files of nginx and apache2 should be modified properly.

• Enable cron. Since I have disabled cron (on a test site), I have to enable it again by un-commenting it on /etc/cron.d/drupal7.
• Replace test settings with live settings. Export drupal setting on the main site with modules/features/save-private-vars.sh and then import them on the new site with drush php-script restore-private-vars.php.
• On the DNS server I change the record of l10n.fs.al to point to the new IP. But the DNS change may take about 2 days to be propagated worldwide. So, after 2-3 days I do again a transfer of users and contributions from the old server to the new one. These transfer operations are designed to be idempotent, which means that the result will be the same even if they are applied many times.