Mirror of the Rel4tion website/wiki source, view at <http://rel4tion.org>

[[ 🗃 ^yEzqv rel4tion-wiki ]] :: [📥 Inbox] [📤 Outbox] [🐤 Followers] [🤝 Collaborators] [🛠 Commits]

Clone

HTTPS: git clone https://vervis.peers.community/repos/yEzqv

SSH: git clone USERNAME@vervis.peers.community:yEzqv

Branches

Tags

master :: maint / admin / ikiwiki /

Git_Server.mdwn

Web editing works now, but the only way to make changes via git is to push to the local repository stored in the ikiwiki user’s home directory. We are going to decouple the git repository from ikiwiki’s clone - regardless of whether they are physically on the same machine or not. Moving them around later will be easy, because the setup doesn’t rely on them being on the same machine, and doesn’t require them to be remote.

By default, the bare master repository uses a post-update git hook, which causes it to update ikiwiki’s clone whenever someone pushes an update. Ikiwiki’s clone is configured to use the bare repo as its remote “origin”, i.e. when a change is made via the web interface, it is pushed to the bare repo.

In our new setup, the bare repo will not touch ikiwiki’s clone: Instead, it will send an HTTP ping, which will cause ikiwiki to pull updates from the bare repo and regenerate HTML. And ikiwiki will access the bare repo using SSH, rather than directly using a file-based URI. Settings domain names to “localhost” allows this setup to work for a single machine setup, while remaining very flexible.

I assume you already have a git server. If you don’t, see the git server tutorial. I will also assume both repsitories are on the same machine, so the domain name used is ‘localhost’. If you use separate remote machines, replace ‘localhost’ with your IP addresses or domain names. A special case is I2P: You’ll need to create tunnels for it, so use ‘localhost’ anyway. I will use my tunnel ports here:

If you don’t use I2P, use your domain name (e.g. ssh://git@git.hello-world.net/wiki.git) for SSH (the port will be 22, but it’s the default so you don’t need to specify it) and the same (e.g. git://git.hello-world.net/wiki.git) for git daemon (the port will be 9418, but it’s the default so you don’t need to specify it). The domain in this example is “git.hello-world.net” but can also be “localhost” if ikiwiki and the git server are on the same machine.

First, we will need to give the ikiwiki user an SSH key, with which it will pull and push to the bare repository located on the git server:

# su - ikiwiki
$ ssh-keygen -t rsa

It will ask several questions. Use the default name for the key. Also it is important that you use a blank password, i.e. just press Enter when asked for a password. It is required because the key will be used from a script.

Now create a new bare repository on the git server. The git server tutorial explains how to do it with Gitolite. Assume we call the new repository ‘wiki’. Give the admin user RW+ access to it. You can give R access to ‘daemon’ and provide a description, so that it is accessible via git:// protocol and appears on Gitweb. Commit the push the change to the git server.

The next step is to add the ikiwiki user. First, copy ikiwiki’s public SSH key (/home/ikiwiki/.ssh/id_rsa.pub) to the git server, i.e. add a new user with key ‘ikiwiki.pub’. The git server tutorial explains exactly how. In the same commit we will also give it the access it needs: Give user ‘ikiwiki’ RW access to the ‘wiki’ repo. Now commit and push.

Now we need to tell ikiwiki to use ‘wiki’ as its remote origin, instead of MyWiki.git which ikiwiki generated as a default. As the ikiwiki user, open file /home/ikiwiki/MyWiki/.git/config. It should look more or less like this:

[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[remote "origin"]
	url = /home/ikiwiki/MyWiki.git
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
	merge = refs/heads/master
	remote = origin

See the ‘url’ setting? We need to change it to point to the new repository we created on the git server. In my case, with I2P as described earlier, the new value is ssh://git@localhost:8950/wiki.git. ‘git’ is the name of the git user on the git server, ‘localhost’ is used because I have a local client tunnel pointing to the git server, 8950 is the port of the local tunnel and ‘wiki.git’ is there because we called the new repository ‘wiki’. Gitolite appends ‘.git’ to the names of repositories (this is a convention for bare repositories).

After changing the ‘url’ and saving the file, it’s time to test the configuration. We’re not done yet, but let’s make sure ikiwiki can pull and push to the new repository.

As the ikiwiki user, cd to the srcdir repo - in our case /home/ikiwiki/MyWiki - and try to git push origin master. You will probably be asked about the authenticity of the SSH server - check the fingerprint if you want ([[TODO|TODO/OPEN]] how?). Once you accept the SSH server, the push operation should work. A new ‘master’ branch will be created on the ‘wiki’ repo.

# su - ikiwiki
$ cd MyWiki
$ git push origin master

Now try pulling. It should say “everything is up to date”. On your personal computer - assuming you gave yourself RW (or RW+) access to ‘wiki’ - clone the ‘wiki’ repository, make some trivial change to the index page (index.mdwn should already be in the repository because you made edits via the web interface earlier) and push them. Now, as the ikiwiki user on the machine where ikiwiki is located, try git pull to make sure you receive the change.

$ git pull

Both pulling and pushing should work without user interaction - no password requested, etc. They both will be used from a script, so it’s important.

As the ikiwiki user, open the setup file - in our case /home/ikiwiki/MyWiki.setup. Under the git plugin settings, find the line defining ‘git_wrapper’. It should look more or less like this:

# git plugin
# git hook to generate
git_wrapper: /home/ikiwiki/MyWiki.git/hooks/post-update
# shell command for git_wrapper to run, in the background
#git_wrapper_background_command: git push github

Since the bare repository will not be pushing to ikiwiki anymore, disable the git_wrapper by prepending a ‘#’ sign to its line (this will turn it into a comment).

Now, tell ikiwiki to push to the ‘wiki’ repository when a change is done via the web interface by uncommenting the git_wrapper_backgroud_command line and setting the command to git push. This section of the file should now look like this:

# git plugin
# git hook to generate
#git_wrapper: /home/ikiwiki/MyWiki.git/hooks/post-update
# shell command for git_wrapper to run, in the background
git_wrapper_background_command: git push

In order to tell ikiwiki to listen to pings from the git server and rebuild the web pages, enable the ‘pingee’ plugin by adding it under the add_plugins variable. It should look more or less like this:

add_plugins:
- goodstuff
- websetup
- pingee

Now rebuild the wiki, so it takes the new setup into account:

$ ikiwiki --setup MyWiki.setup --rebuild --verbose

Once side of our new setup now should work: Whenever a change is made to the wiki via the web interface, it should push it to the git server. You can test it by making a change, and then pulling in your personal clone and make sure you received the change. The other direction doesn’t work yet: Ikiwiki can pull, but it doesn’t know when. We are going to tell the git server to ping ikiwiki whenever a change is made via a personal clone (i.e. not the web interface).

On the git server machine, as the git user, go to the hooks folder of the ‘wiki’ repository. A typical path would be /home/git/repositories/wiki.git/hooks. Create a new file named post-receive. This file is a script which will be executed whenever a new commit it received. The first line should be the shebang !#/bin/sh, then you can add a comment (a comment line starts with #) and finally the command itself. We’ll need to tell the git server to send a ping to ikiwiki.

If you use I2P or Tor, you need to tell the git server to use a proxy to access ikiwiki, because otherwise it cannot access .onion websites or .i2p eepsites. If you use the typical port 4444 as the I2P HTTP proxy, you can set the proxy like this:

export http_proxy="http://localhost:4444/"

If you use privoxy (I use it), specify the privoxy port instead (usualy 8118):

export http_proxy="http://localhost:8118/"

If you use the clearnet (regular internet), you don’t need the proxy.

Now, the command itself. Basically, we’ll use a call to ‘wget’ (time to install it if you haven’t yet) which will trigger the ping, and direct the returned result to /dev/null because we don’t need it - when you push from a personal clone, you’ll still see the feedback from the git server, so you’ll know the ping was sent successfully. The wget invocation looks like this, with the URL being the public URL of the ikiwiki instance:

wget "http://partager.i2p/ikiwiki.cgi?do=ping" -O /dev/null

However, this is not enough: We need to ping ikiwiki only if the commit was made from somewhere else, otherwise we will cause ikiwiki to pull again, while the git server isn’t done updating its own repository yet with the commit sent by ikiwiki. There are two ways to check where the commit came from:

The second option is better if you use gitolite, because the first one has a corner case where it may not work: If you push commits you created on a local ikiwiki clone via local CGI, the author email would contain ‘@web’ as well. If this use case is relevant to you, you can try to cross the author email with the committer email. Maybe it will then cover this case - I didn’t try.

For the first case, the author test looks like this:

git log -1 --format=format:%ae HEAD | grep -e '@web$' -e 'ikiwiki@HOST'

Just change HOST to the hostname of the machine where ikiwiki runs. The ‘ikiwiki’ before the @ is the name of the ikiwiki user we created. To be sure, you can check the author of the first commit made to your wiki. This should be the ikiwiki user. You can determine this by running git log --format=format:%ae HEAD on any wiki clone (or on the ‘wiki’ bare repo itself) and looking at the last email in the list.

For the second case, the test looks like this:

[ x"$GL_USER" = x"ikiwiki" ]

“ikiwiki” here is the name of the Gitolie-registered user who made the commit. The user name comes from the filename of the SSH key. For example, if you added ikiwiki’s SSH key to Gitolite as ‘ikiwiki.pub’, the username to test here is “ikiwiki”.

Now use || to combine the test and the ping: If the commit does not come from ikiwiki, then send the ping. Using the first case, it looks like this (again, replace HOST with the correct hostname):

git log -1 --format=format:%ae HEAD | grep -e '@web$' -e 'ikiwiki@HOST' || \
wget "http://partager.i2p/ikiwiki.cgi?do=ping" -O /dev/null

And for the second case it looks like this:

[ x"$GL_USER" = x"ikiwiki" ] || wget "http://partager.i2p/ikiwiki.cgi?do=ping" -O /dev/null

Here’s a full example, using Gitolite (second case) and an I2P address for pinging accessed via Privoxy:

#!/bin/sh

## special hook for ikiwiki repo:
## update the srcdir clone (which will regenerate HTML),
## if a commit not-from-ikiwiki was received
## (if it's from ikiwiki, it already has it so no need
##  to update it... that would cause a deadlock)

## "ikiwiki" here is the gitolite-registered user (i.e.
## username determined by SSH key name, e.g. ikiwiki.pub)
## from which the push came

export http_proxy="http://localhost:8118/"
[ x"$GL_USER" = x"ikiwiki" ] || wget "http://partager.i2p/ikiwiki.cgi?do=ping" -O /dev/null

Save the file. git won’t use it unless it is executable, so we need to update permissions. These are the command for the last step, and the last command is the permission change (makes our hook executable):

# su - git
$ cd repositories/wiki.git/hooks
$ nano post-receive
$ chmod 750 post-receive

Time to test the new setup. Using a personal clone, push change and make sure the HTML gets updated automatically. Then, just to be sure, you can try again to make a web edit and make sure it got pushed by pulling in my personal clone and seeing the update.

Done. You can delete the old bare repository ikiwiki created, to avoid confusion later:

# su - ikiwiki
$ rm -rf MyWiki.git

Congratulations! You have a new working ikiwiki!

[See repo JSON]