Revision as of 21:38, 18 March 2024

Created on 2024-03-14 for the HSL install on AWS.
Previous Version: MediaWiki_Install_2022-09-01

Notes

Change "heatsynclabs.wiki" to Your Identifier

I'm using "heatsynclabs.wiki" here because that's where I'm deploying. Anywhere you see "heatsynclabs.wiki" below you should change that to suit your case.

Add Markdown Handler

https://www.mediawiki.org/wiki/Extension:WikiMarkdown

Create a Host

I'm not going to document this part. Pick your favorite hosting service and create a Debian 12 host.

If your provider supports it, get a permanent IP address and point it at the new host. This will let you blow away the host, create a new one, point the IP at the new host, and redo the install without having to update the DNS entry and wait for the cascade.

If your provider has default-deny firewalling, make sure ports 22, 80, and 443 are open, which will allow SSH, HTTP, and HTTPS, respectively.

Note the IP address.

update your /etc/hosts:

35.165.2.213    heatsynclabs.wiki
2600:1f14:4b8:3800:524:b173:6c45:549e    heatsynclabs.wiki

update .ssh/config on your client box:

Host heatsynclabs.wiki
  HostName 35.165.2.213
  User admin
  IdentityFile /path/to/key.pem

System Update and Core Software

Following is based, in part, on MediaWiki Installation Requirements.

Log in to your host.

Create the setup script:

$ mkdir bin
$ cat > bin/system-update.sh
sudo apt update -y && \
sudo apt upgrade -y && \
sudo apt install -y emacs-nox && \
sudo apt install -y apache2 certbot python3-certbot-apache && \
sudo apt install -y mariadb-server php php-mysql libapache2-mod-php && \
sudo apt install -y php-xml php-mbstring php-apcu php-intl php-cli php-curl && \
sudo apt install -y git imagemagick texlive composer

Control-d will exit and save the file.

$ chmod ug+x bin/system-update.sh
$ ./bin/system-update.sh

If you're not sure that it all ran correctly, you can safely re-run the script. You'll know it ran correctly if the output is roughly 40 lines saying things like, "already the newest", "0 upgraded", and "... Done".

MediaWiki Tarball

I am installing from the tarball instead of using the Debian package because other package tools, like RPM or Snap, or other distros than Debian, might put directories in different places. Following the MediaWiki tarball instructions should be close to universal.

$ mkdir apps
$ cd apps
$ wget https://releases.wikimedia.org/mediawiki/1.41/mediawiki-1.41.0.tar.gz

One difference on other distros may be the location of web content directories. Debian puts them in /var/www. Adjust the following commands to match your distro's structure if you're not using Deb 10.

I'm moving the stock Apache site to a directory for static content, and linking mediawiki to the wiki engine directory. The symlink will make it a bit easier to upgrade to the next version of MediaWiki.

$ cd /var/www
$ sudo tar -xvzf ~/apps/mediawiki-1.41.0.tar.gz
$ sudo mv html static
$ sudo ln -s mediawiki-1.41.0 mediawiki

Configure MariaDB

Pick a username for MediaWiki to use (I'm using wiki_wiki as an example).

Pick a database name (I'm using hsl_wiki as an example). We'll also create "install_wiki" to hold the default install as a backup in case we have troubles cloning the legacy database.

Pick a password other than "CHANGE THIS PASSWORD".

$ cd ~/bin
$ sudo cat > mariadb-wiki-account.sql
create database install_wiki;
create database hsl_wiki;
grant all on install_wiki.* to 'wiki_wiki'@'localhost' identified by 'CHANGE THIS PASSWORD';
grant all on hsl_wiki.* to 'wiki_wiki'@'localhost' identified by 'CHANGE THIS PASSWORD';
flush privileges;

Control-D to write the file.

$ sudo mariadb < mariadb-wiki-account.sql

Then you can verify it worked if you like. (there won't be any tables, but it shouldn't give you an auth error)

$ mariadb -u wiki_wiki -p
MariaDB> show tables in traxel_wiki;
MariaDB> exit

Configure DNS

This is assuming that there is a legacy wiki running on wiki.domain.com, and you will be bringing up the new machine with the temporary name wiki-new.domain.com. Once you have the new machine running and validated, you will migrate wiki.domain.com to the new machine.

Go to your domain name registrar and add an A and AAAA Record for the machine and three CNAME records; one for the root, one for all hosts, and one for static. The static one isn't strictly needed (covered by *). I'm using "mediawiki01" as the machine name, you can use whatever you like.


Type	Host	Value	TTL
A Record	mediawiki01	35.165.2.213	Automatic
AAAA Record	mediawiki01	2600:1f14:4b8:3800:524:b173:6c45:549e	Automatic
CNAME	*	mediawiki01.heatsynclabs.wiki	Automatic
CNAME	@	mediawiki01.heatsynclabs.wiki	Automatic
CNAME	static	mediawiki01.heatsynclabs.wiki	Automatic

It may take a bit to cascade through your resolvers. If you want to be able to hit it immediately, you can update your /etc/hosts file. Add a line like "35.165.2.213 mediawiki.domain.com" but with your IP and domain, after the localhost mappings.

Configure Apache

Next we'll add an Apache config for wiki-new.domain.com that points to the MediaWiki engine, and another for wiki-static-new.domain.com that points to a static content directory.

$ cd ~/
$ mkdir -p conf/apache
$ cd conf/apache

/etc/apache2/sites-available/wiki.conf

$ cd ~/conf/apache
$ cat > wiki.conf
<VirtualHost *:80>
  ServerName heatsynclabs.wiki
  ServerAlias mediawiki01.heatsynclabs.wiki

  ServerAdmin webmaster@localhost
  DocumentRoot /var/www/mediawiki

  ErrorLog ${APACHE_LOG_DIR}/wiki-error.log
  CustomLog ${APACHE_LOG_DIR}/wiki-access.log combined
</VirtualHost>

/etc/apache2/sites-available/wiki-static.conf

$ cd ~/conf/apache
$ cat > wiki-static.conf
<VirtualHost *:80>
  ServerName static.heatsynclabs.wiki

  ServerAdmin webmaster@localhost
  DocumentRoot /var/www/static

  ErrorLog ${APACHE_LOG_DIR}/wiki-static-error.log
  CustomLog ${APACHE_LOG_DIR}/wiki-static-access.log combined
</VirtualHost>

/etc/apache2/sites-available/000-default.conf

$ emacs -nw /etc/apache2/sites-available/000-default.conf
<VirtualHost *:80>
  ServerAdmin webmaster@localhost
  DocumentRoot /var/www/mediawiki

  ErrorLog ${APACHE_LOG_DIR}/error.log
  CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Copy to sites-available

$ sudo cp ~/conf/apache/*conf /etc/apache2/sites-available/

Activate Host Configs

Activate the mediawiki conf and bounce the server.

$ sudo a2ensite wiki wiki-static
$ sudo service apache2 restart

Check Sites

Now you should be able to see the two Apache configs at the desired host addresses:

http://wiki-new.traxel.com/ (should show the MediaWiki logo page with a link to the config script)
http://wiki-static-new.traxel.com/ (should show the static Apache2 root directory)

If you went through this script quickly, the DNS may not have cascaded yet. You could try editing your /etc/hosts file, or go get a cup of coffee and read Reddit for half an hour, then try again.

Enable TLS (SSL)

To keep your errors easy to diagnose and resolve, do one site at a time and verify / fix errors independently.

$ sudo certbot --apache -d wiki-new.traxel.com

When it asks about redirects, choose, "2: Redirect - Make all requests redirect to secure HTTPS access."

Hit http://wiki-new.traxel.com and verify it redirects to https:// and shows the correct content. If there are any errors, fix them before moving on.

$ sudo certbot --apache -d wiki-static-new.traxel.com

When it asks about redirects, choose, "2: Redirect - Make all requests redirect to secure HTTPS access."

Hit http://wiki-static-new.traxel.com and verify it redirects to https:// and shows the correct content. Fix any errors before moving on.

Run The MediaWiki Config Script

Get a logo image file ready to drag and drop. You will need it later.

Go to https://wiki-new.traxel.com/ and click on the setup hyperlink.

Connect to Database

database host: localhost
database name: install_wiki
database table prefix: <LEAVE BLANK>
database username: wiki_wiki
database password: <YOUR PASSWORD>

Database Settings

I use the same settings for web access as for installation.

Name

name of wiki: Traxel Wiki
project namespace: same as the wiki name
your username: <YourWikiname, conventionally FirstnameLastname>
password: <A DIFFERENT PASSWORD>
repeat password: <THE SAME, DIFFERENT PASSWORD>
email address: <LEAVE BLANK>
ask me more questions: tick this radio button
I'm bored already: don't tick this, there's important stuff ahead

Options

In the sections below from Special Pages to Other, you can get more info on extensions by tacking the extension name after the colon in the following URL: https://www.mediawiki.org/wiki/Extension:TextExtracts

user rights profile: Authorized editors only
copyright and license: No license footer
email settings: uncheck Enable outbound email
skins: select all and use Vector as the default
special pages: I include all 5
editors: I include all 3
parser hooks: I include CategoryTree, Cite, Math, SyntaxHighlight_GeSHi, and TemplateData
media handlers: I include PdfHandler
spam prevention: I don't use these, since I used "authorized editors" above.
api: I include PageImages
Other: i include MultimediaViewer and SecureLinkFixer
images and file uploads: check "enable file uploads", leave the default directory setting
personalization: Drag your wiki logo to both the "Logo" and "Sidebar Logo" settings. (Note: it is a *lot* easier to do this now, don't skip this step)
advanced configuration: use PHP caching - we installed apcu earlier.

Language

Pick your language.

Install

When you click continue, this page will write the database.

Complete

This page will automatically start a download of the LocalSettings.php file.

Install LocalSettings.php

Push a copy to the server at ~/conf/LocalSettings-[datestamp].php .

Log in to the server to install the config. While we're at it, we'll set the images directory structure to be owned by www-data, so image uploads will work.

$ sudo usermod -aG www-data admin
$ sudo chown -R admin /var/www/mediawiki/
$ exit

Then log back in.

$ groups # should show www-data
$ cp ~/conf/LocalSettings-traxel-2022-09-05.php /var/www/mediawiki/LocalSettings.php
$ sudo chown -R www-data /var/www/mediawiki/images
$ sudo service apache2 restart

Now the wiki should be live.

https://wiki-new.traxel.com/

Configure Markdown

ChatGPT makes its output available in Markdown format. While I personally prefer MediaWiki format, Markdown is both more broadly supported and is what ChatGPT spits out; it's handy to support it directly. Fortunately, there's a MediaWiki plugin for Markdown.

Composer

Earlier, during the initial system setup, we included the following in support of the Markdown plugin:

apt install -y composer

Now we need to enable it in the wiki config:

$ cd /var/www/mediawiki
$ sudo cp composer.local.json-sample composer.local.json
$ sudo emacs -nw composer.local.json

Add the following line in the extra plugin include:

{
	"extra": {
		"merge-plugin": {
			"include": [
				"extensions/WikiMarkdown/composer.json"
			]
		}
	}
}

$ composer update
$ cd extensions
$ git clone https://github.com/kuenzign/WikiMarkdown.git

= Bookmark: Paused Here on HSL Wiki =
This is where I stopped on the HSL wiki install on 2024-03-18. Cloning is probably going to use an export/import approach instead of what is shown below, due to the extreme version difference and probable incompatibility.

= Clone the Legacy Wiki =
== LocalSettings.php ==

Compare the LocalSettings.php from the legacy server with the new LocalSettings.php file.

I used diff and did a line by line check. It was pretty quick and all the differences were reasonable.

I wasn't satisfied with the permissions from my legacy wiki, nor what was produced by the configuration wizard. I updated the permissions section in LocalSettings.php to read as follows:

<pre>
# The following permissions were set based on your choice in the installer
$wgGroupPermissions['*']['createaccount'] = true;
$wgGroupPermissions['*']['edit'] = false;
$wgGroupPermissions['user']['edit'] = false;
$wgGroupPermissions['user']['move'] = false;
$wgGroupPermissions['writer']['edit'] = true;
$wgGroupPermissions['writer']['move'] = true;

Also add this at the end of the file:

$wgDBerrorLog = "/var/log/mediawiki/db-errors.log";

And:

$ sudo mkdir /var/log/mediawiki
$ sudo chown www-data /var/log/mediawiki

Database

Log in to the legacy server, stop Apache, and dump the database.

$ sudo service apache stop
$ mysqldump -p -u wiki_wiki traxel_wiki | bzip2 > traxel-wiki-dump-2022-09-05.sql.bz2

Transfer the database dump to the new server.

Load the database dump into the new server.

$ bzip2 -dc traxel-wiki-dump-2022-09-05.sql.bz2 | mariadb -p -u wiki_wiki traxel_wiki

Edit /var/www/mediawiki/LocalSettings.php. Change $wgDBname = "install_wiki"; to $wgDBname = "traxel_wiki"; then bounce Apache.

$ sudo emacs -nw /var/www/mediawiki/LocalSettings.php
$ sudo service apache2 restart

Images

Back up the images from the legacy wiki. Log in to the old server.

$ cd /var/www/mediawiki
$ tar -cvjf ~/images.tar.bz2 ./images

Push a copy of the tarball to the new server at ~/data/.

Log in to the new server to load the images.

$ sudo service apache2 stop
$ cd /var/www/mediawiki
$ sudo mv images images.old
$ sudo tar -xvjf ~/data/images.tar.bz2
$ sudo chown -R www-data images

Test The Wiki

Check out the wiki, make sure everything is working. The next step is the cutover; if anything isn't working right, it's easy to fix now.

Spot check content pages.
Spot check pages that contain images.
Create an account, but don't give it privileges.
Try editing a page using your admin account. (should work)
Try editing a page using the unpermissioned account. (should fail)
From your admin account, grant permissions to the unpermissioned account.
Try editing a page using the previously unpermissioned account.

Switch DNS

Go to your DNS provider and point wiki and wiki-new to your new server.

Also update LocalSettings.php; change $wgServer = "https://wiki-new.traxel.com"; to $wgServer = "https://wiki.traxel.com";

$ sudo emacs -nw /var/www/mediawiki/LocalSettings.php

Now it's going to take a while for the DNS to cascade before we can do the new TLS certs. That makes this a good time to set up the backups.

Set Up Nightly Backups

This script will make the backup available on the static content site. Until the DNS cutover, you can see it at https://wiki-static-new.traxel.com/backups Later it will be available at https://wiki-static.traxel.com/backups .

This assumes that the administrator user account is "admin".

In the script below, put your password on the first line, but don't change "YOUR_PASSWORD" in the sed line near the end. That is used to overwrite your password before creating the backup archive.

$ sudo mkdir -p /var/www/static/backups
$ sudo chown admin /var/www/static/backups
$ cat > ~/bin/nightly_backup.sh
export db_pass=[YOUR_PASSWORD]
export wiki_db_name=traxel_wiki
export dstamp=`date +%Y-%m-%d`
export bundle_dir=wiki-backup-$dstamp
export bundle_path=/tmp/$bundle_dir
export dump_path=/tmp/wiki-backup-$dstamp/$wiki_db_name.sql.bz2
export image_archive_path=/tmp/wiki-backup-$dstamp/$wiki_db_name-images.tar.bz2
export conf_path=/tmp/wiki-backup-$dstamp/LocalSettings.php
export tarball_path=/var/www/static/backups/wiki-backup.tar.bz2
mkdir -p $bundle_path
cp /var/www/mediawiki/LocalSettings.php $conf_path
sed -i s/$db_pass/YOUR_PASSWORD/ $conf_path
mysqldump -u wiki_wiki --password=$db_pass $wiki_db_name | bzip2 > $dump_path
tar -C /var/www/mediawiki -cjf $image_archive_path /var/www/mediawiki/images
tar -C /tmp -cvjf $tarball_path $bundle_dir

Control-d will exit and save the file.

Try running it and verify everything works.

$ cd ~
$ chmod ug+x bin/nightly_backup.sh
$ ./bin/nightly_backup.sh

Verify that the database password is getting replaced with "YOUR_PASSWORD" in LocalSettings.php.

Try going to https://wiki-static-new.traxel.com/backups to confirm the backup is visible.

Running it repeatedly will just overwrite the files, so you're safe to test to your heart's content without filling the hard drive.

Cron Entry

Start with a crontab entry that runs the backup every ten minutes, so you can see it run.

*/10 * * * * /home/admin/bin/nightly_backup.sh

Then switch it to nightly. My server runs on zulu time, so I set it to run at 01:20 with the following:

20 8 * * * /home/admin/bin/nightly_backup.sh

Get New TLS Certificates

Hopefully the backup setup has taken long enough that we can now get the production URL TLS certs.

$ sudo certbot --apache -d wiki-static.traxel.com -d wiki.traxel.com
$ sudo service apache2 restart

Pull The Backups Somewhere

Backups don't do much good if they're not being stored somewhere. Set up a cron script, maybe on one of HSL's on-prem servers, to periodically pull a copy of the wiki backup.

Grab this URL periodically, perhaps with a rotation cycle that retains dailies for a week, weeklys for a month, monthlys for a year, and yearlys forever (or something).

I'm using the following, which is simpler but has a bit more risk of loss:

30 9 * * * wget -O /home/bob/Downloads/wiki-backups/traxel-wiki-nightly.tar.bz2 https://wiki-static.traxel.com/backups/wiki-backup.tar.bz2
30 9 1 * * wget -O /home/bob/Downloads/wiki-backups/traxel-wiki-monthly.tar.bz2 https://wiki-static.traxel.com/backups/wiki-backup.tar.bz2
30 9 1 1 * wget -O /home/bob/Downloads/wiki-backups/traxel-wiki-yearly.tar.bz2 https://wiki-static.traxel.com/backups/wiki-backup.tar.bz2

Test Everything

Errors & Fixes

Error Listing All Pages

Tue Sep 6 4:02:42 UTC 2022	ip-172-26-9-72	traxel_wiki	Error 1176 from SpecialPrefixindex::showPrefixChunk, Key 'page_name_title' doesn't exist in table 'page' (localhost) SELECT  page_namespace,page_title,page_id,page_namespace,page_title,page_is_redirect,page_is_new,page_latest,page_touched,page_len,page_restrictions,page_content_model  FROM `page`  FORCE INDEX (page_name_title)  WHERE page_namespace = 0 AND (page_title LIKE 'Trax%' ESCAPE '`' ) AND (page_title >= '')  ORDER BY page_title LIMIT 346   localhost
#0 /var/www/mediawiki-1.38.2/includes/libs/rdbms/database/Database.php(1564): Wikimedia\Rdbms\Database->getQueryExceptionAndLog(string, integer, string, string)
#1 /var/www/mediawiki-1.38.2/includes/libs/rdbms/database/Database.php(1173): Wikimedia\Rdbms\Database->reportQueryError(string, integer, string, string, boolean)
#2 /var/www/mediawiki-1.38.2/includes/libs/rdbms/database/Database.php(1810): Wikimedia\Rdbms\Database->query(string, string, integer)
#3 /var/www/mediawiki-1.38.2/includes/libs/rdbms/database/DBConnRef.php(69): Wikimedia\Rdbms\Database->select(string, array, array, string, array)
#4 /var/www/mediawiki-1.38.2/includes/libs/rdbms/database/DBConnRef.php(319): Wikimedia\Rdbms\DBConnRef->__call(string, array)
#5 /var/www/mediawiki-1.38.2/includes/specials/SpecialPrefixindex.php(205): Wikimedia\Rdbms\DBConnRef->select(string, array, array, string, array)
#6 /var/www/mediawiki-1.38.2/includes/specials/SpecialPrefixindex.php(103): SpecialPrefixindex->showPrefixChunk(integer, string, string)
#7 /var/www/mediawiki-1.38.2/includes/specialpage/SpecialPage.php(671): SpecialPrefixindex->execute(NULL)
#8 /var/www/mediawiki-1.38.2/includes/specialpage/SpecialPageFactory.php(1378): SpecialPage->run(NULL)
#9 /var/www/mediawiki-1.38.2/includes/MediaWiki.php(315): MediaWiki\SpecialPage\SpecialPageFactory->executePath(string, RequestContext)
#10 /var/www/mediawiki-1.38.2/includes/MediaWiki.php(912): MediaWiki->performRequest()
#11 /var/www/mediawiki-1.38.2/includes/MediaWiki.php(563): MediaWiki->main()
#12 /var/www/mediawiki-1.38.2/index.php(53): MediaWiki->run()
#13 /var/www/mediawiki-1.38.2/index.php(46): wfIndexMain()
#14 {main}

1.32 version of table "page":

  UNIQUE KEY `page_name_title` (`page_namespace`,`page_title`),

1.38 version of table "page":

  UNIQUE KEY `name_title` (`page_namespace`,`page_title`),

This fixes it:

create unique index page_name_title on page (page_namespace, page_title);
drop index name_title on page;

MediaWiki Install: Difference between revisions