Using the Git Stash command

The git stash command is used to store changes to a dirty git tree when pulling changes[ref]http://man.github.com/git/git-stash.html[/ref].

For example, if you are working on a repository, but are not ready to commit your changes, you may run.

git stash

This will store the changes you have made since the last revision and allow you to start back fresh at the point of your last commit.

Once you are done, you may run

git stash pip

That will restore your repository back to where you were when you ran git stash.

Git: Add all remote branches

Adding each remote branch to a local git repository sometimes can be a pain. IF there are many, you have to repeat your self over and over. Here is a quick, copy and past drop into you console, way to add all the remote branches to your local repository.

for b in `git remote show origin |grep tracked |awk '{print $1}'`
do
    LOCALBRANCH=`git branch |sed 's/* //g' |sed 's/  //g' |grep $b`
    if [ "$LOCALBRANCH" != "$b" ]; then
        git branch -t $b origin/$b
    fi
done

Once your done, you should still be in your original branch were you started. You will still need to update each branch by it self. You may also use something like git-up to update all the branches at once.

Gallery3 Installed on a Nginx Server

A few months ago I converted my web cluster from Apache to Nginx. Initially I was only concerned with my WordPress sites as they get, by far the most traffic. Their conversion went well, with only very minor issues.

Since I was happy with the results, I moved to quickly to my MediaWiki site. It’s conversion went very well and actually end up allowing the site to accept requests with and without index.php in them.

During this time, my other random sites, using odd or old software, was simply proxied back to the still running Apache install on my server. Using nginx’s proxy configuration, i was able to just change the port Apache listen on and left the old configurations as they were.  This allowed me to stay up, but didn’t give me any of the benefits of nginx.  Images from the my Gallery3 gallery were still being served from Apache, threw Nginx.

After getting everything working, I decided to start on Gallery3, since Gallery3 uses .htaccess files to secure images, I wasn’t sure how I would be able to go about this.  After some investigating I found that Gallery3 only uses the .htaccess files to block downloading of the actual images them self’s, the php pages are still secured threw normal permissions.

Configuring Nginx

Setting up Gallery3 under Nginx is pretty straightforward, when ignoring the .htaccess file talked about above.  As with all my Nginx setup’s, im using php-fpm via a UNIX sock.  Switching to a TCP connection (such as 127.0.0.0:8000), can be substituted with only minor tweaks.

To start out, create a new file in your nginx configuration directory, named gallery3.conf.  This file will be generic enough that you will be able to use it for any Gallery3 install on the server. Your site specific information will live in your main nginx.conf file.

Create a new file named gallery3.conf, and copy the below into it:

location ~* .(js|css|png|jpg|jpeg|gif|ico|ttf)$ {
    expires 180d;
#    if_modified_since off;
#    add_header Last-Modified "";
}

if (!-e $request_filename) {
    rewrite ^/(.+)$ /index.php?kohana_uri=$1 last;
}

location /var/ {
    try_files $uri /index.php?kohana_uri=$2;
}

location = /downloadalbum/zip/album/1 {
    return 404;
}

location  ~* .php$ {
    include fastcgi_params;
    fastcgi_index  index.php;
    fastcgi_split_path_info ^(.+.php)(.*)$;
    fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
    fastcgi_param  PATH_INFO        $fastcgi_path_info;
    fastcgi_pass php;

    index index.php;

    if (-f $request_filename) {
        rewrite ^/var/albums/(.*)$ /file_proxy/$1 last;
        expires max;
        break;
    }

    if (!-e $request_filename) {
        rewrite ^/(.+)$ /index.php?kohana_uri=$1 last;
    }
}

Now you need to setup your new site in Nginx’s main config file.  Below is the most basic setup, it assumes you have a running Nginx server and are only adding this site to it.  Note the name of the server and the root location of the files needs to be updated.

Added to nginx.conf:

server {
    listen 80;
    server_name gallery.example.com;
    root /var/www/gallery.example.com;
    include gallery3.conf;
}

This next part really sucks and I wish there was a way around it, in your Gallery3 installation, go into applications/config/config.php and modify the “index_page” setting as shown below.

application/config/config.php:

$config["index_page"] = "";

What sucks about this is you will need to do it after EVERY UPDATE. If after you update Gallery3 and you loose your style sheet and java scripts, this is why.

The config file should not live in a location that get’s updated automatically with new versions of software.

Installing WordPress on Nginx

I, like most people, started out by using Apache and really didn’t see anything wrong with it.  It’s relatively easy to setup, it’s used by most sites so support is a snap and a default and it’s already installed on most distributions. The thing is, it’s slow, the easy of use is paid for by speed.

So this is where Nginx comes in.  Nginx is not the easiest software to setup.  It requires you to tell it what different file types you will be using and how to handle them.  It requires you to tell it where scripts live and where static files live.  It also requires you to use an external php server, such as FastCGI.

PHP-FPM Setup

Nginx does not provide FastCGI for you (FastCGI is what your web server uses to interact with WordPress’s PHP code), so you’ve got to have a way to spawn your own FastCGI processes.

My preferred method is using of running FastCGI is with php-fpm.  Since I’m using Fedora, and there is a yum packet already built for php-fpm, it’s quick and easy to install.

Installing php-fpm is pretty straightforward:

yum install php-fpm

After installing php-fpm you have to start it. The rpm for php-fpm installs the service script for you, you only need to enable starting at boot, and start the process.

chkconfig php-fpm on
service php-fpm start

Nginx

The next part is to install Nginx on your server.  This is as straightforward as installing php-fpm on Fedora, when using yum.

yum install nginx

Once Nginx is installed, you need to set it up to server your site.

Configuring Nginx for WordPress

So we now have the needed software installed, next we need to set it all up. Below is the config for a standard, simple WordPress site named example.com.
nginx.conf:

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] $http_host "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;
    sendfile        on;
    #tcp_nopush     on;
    rewrite_log     on;
    keepalive_timeout  5;
    index              index.php index.html index.htm;

    # Upstream to abstract backend connection(s) for PHP.
    upstream php {
        server unix:/var/run/php-fpm.socket;
    }

    server {
        listen 80;
        server_name example.com;
        server_name www.example.com;
        root /var/www/example.com;

        if ($http_host != "example.com") {
                rewrite ^ http://example.com$request_uri permanent;
        }

        include wordpress.conf;
    }
}

wordpress.conf:

# WordPress single blog rules.
# Designed to be included in any server {} block.

# Deny all attempts to access hidden files such as .htaccess, .htpasswd, .DS_Store (Mac).
location ~ /. {
        deny all;
        access_log off;
        log_not_found off;
}

# Deny access to any files with a .php extension in the uploads directory
location ~* ^/wp-content/uploads/.*.php$ {
        deny all;
        access_log off;
        log_not_found off;
}

# Deny access to any files with a .php extension in the uploads directory for multisite
location ~* /files/(.*).php$ {
        deny all;
        access_log off;
        log_not_found off;
}

# This order might seem weird - this is attempted to match last if rules below fail.
# http://wiki.nginx.org/HttpCoreModule
location / {
        try_files $uri $uri/ /index.php?$args;
}

# Add trailing slash to */wp-admin requests.
rewrite /wp-admin$ $scheme://$host$uri/ permanent;

# Directives to send expires headers and turn off 404 error logging.
location ~* .(js|css|png|jpg|jpeg|gif|ico|ttf)$ {
        expires 180d;
        log_not_found off;
}

# Pass all .php files onto a php-fpm/php-fcgi server.
location ~ .php$ {
        # Zero-day exploit defense.
        # http://forum.nginx.org/read.php?2,88845,page=3
        # Won't work properly (404 error) if the file is not stored on this server, which is entirely possible with php-fpm/php-fcgi.
        # Comment the 'try_files' line out if you set up php-fpm/php-fcgi on another machine.  And then cross your fingers that you won't get hacked.
        try_files $uri =404;

        fastcgi_split_path_info ^(.+.php)(/.+)$;
        #NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini

        include fastcgi_params;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
#       fastcgi_intercept_errors on;
        fastcgi_pass php;
}

RoundCube: Error No [604]

I recently had a client who moved RoundCube to a new server. Since the move they are receiving the below error when they go to their webmail site.

SERVICE CURRENTLY NOT AVAILABLE!
Error No. [604]

After much digging and searching, I wasn’t coming up with much. I finely started walking threw the config file seeing what was enabled and were the different config options were pointing. After some digging I found they had memcache configured for session data. The php-pecl-memcache module wasn’t installed. I installed the module via yum, as it was a Fedora system using the below command.

yum install php-pecl-memcache

Since the config was still pointing at the running memcache server, the site came back up and is now working well again.

Configuring WordPress to use Memcached

When using WordPress self hosted software it’s generally a good idea to cache as much as possible.  Object Caching allows you store parts of the pages in memory for quicker retrieval, since the server will not need to look as much up from the SQL database.

Installing the needed parts

To start out, you will need to have memcached installed on your server. If your using Fedora, you may install memcached via Yum as follows.

yum -y install memcached php-pecl-memcached perl-Cache-Memcached

Configuring memcached

After installing memcached you need to configure it. If working in Fedora, and using the Yum install as talked about above, you will need to change the memcached confuration by modifying it’s sysconfig file located at /etc/sysconfig/memcached.

A default configuration may look like this.

PORT="11211"
USER="memcached"
MAXCONN="1024"
CACHESIZE="64"
OPTIONS="-l 127.0.0.1"

If you would like to share this memcached server with other webservers, change the address from 127.0.0.1 to the server’s actual address.

To set memcached to start automatically when the server get’s rebooted, run:

chkconfig memcached on

And of course, don’t for get to start it

service memcached start

Configuring WordPress

After memcached is installed, you need to configure the WordPress side.

Next you should install the Memcached Object Cache plugin, but be careful, this is not a normal plugin.  You should not activate this plugin as you would with a normal plugin, but instead download it as you normally would, but then you need to copy the object-cache.php file to your wp-content folder.

From the root of your WordPress install, run the following:

cp wp-content/plugins/memcached/object-cache.php wp-content/

Now we need to configure WordPress to use the memcached server. Add the following near the end of your wp-config.php file.

global $memcached_servers;
$memcached_servers = array('default' => array('127.0.0.1:11211'));

Note that were using the same server and port (127.0.0.1:11211) as was configured above while we were setting up memcached.

Checking in on memcached

Memcached is one of those things that just sort of runs. There’s not much direct feed back, besides the speed difference on your site.

One quick way is memcache-top. memcache-top will show you the current status of your memcached server.

To install, run the following.

wget http://memcache-top.googlecode.com/files/memcache-top-v0.6
chmod +x memcache-top-v0.6
./memcache-top-v0.6

Running ./memcache-top-v0.6 will assumed the default configuration we used here.

Dovecot Auto Update Script

Here is a small auto update script for a Fedora build. This script will auto install all needed software and start Dovecot once it’s finished.

#!/bin/bash

DIR=/var/src

if [ ! -e $DIR ]; then
  echo "Creating base directory $DIR"
  mkdir -p $DIR
fi

if [ ! -e $DIR/dovecot-2.0 ]; then
  echo "Dovecot 2.0 is not installed, Downloading a fresh copy of Dovecot 2.0"
  cd $DIR
  hg clone http://hg.dovecot.org/dovecot-2.0/ dovecot-2.0 > /dev/null
  if [ $? != 0 ]; then
    echo "Failed to download a fresh copy of Dovecot 2.0"
    exit 1
  fi
  echo "Installing any needed dependinces"
  yum -y install mercurial gcc autoconf automake libtool perl-Text-Iconv gettext gettext-devel gettext-libs openssl openssl-devel sqlite sqlite-devel zlib zlib-devel > /dev/null
  cd $DIR/dovecot-2.0
else
  cd $DIR/dovecot-2.0
  echo "Downloading updated version of Dovecot 2.0"
  echo -n "Current Version is: "
  dovecot --version
  hg pull -u > /dev/null
  if [ $? != 0 ]; then
    echo "Failed to update the Dovecot 2.0 install"
    exit 1
  fi
  dovecotver=0
  hgrepover=1
  dovecotver=`dovecot --version |awk '{print $2}' |sed 's/(//g' |sed 's/)//g'`
  hgrepover=`hg summary |grep parent |sed 's/:/ /g' |awk '{print $3}'`
  if [ ! $dovecotver ]; then
    dovecotver=0
  fi
  if [ ! $hgrepover ]; then
    hgrepover=1
  fi
  if [ $dovecotver == $hgrepover ]; then
    echo "The installed version and the downloaded version are both `dovecot --version`, not updating."
    exit 0
  else
    echo "The installed version is $dovecotver, but the current version on hg is $hgrepover, updating!"
  fi
  rm -rf *
  hg revert --all > /dev/null
  if [ $? != 0 ]; then
    echo "Failed to restore Dovecot 2.0 install"
    exit 1
  fi
fi

echo "Starting to comple Dovecot 2.0"
./autogen.sh > /dev/null 2> /dev/null
if [ $? != 0 ]; then
  echo "Failed to run autogen.sh for Dovecot 2.0"
  exit 1
fi

./configure --prefix=/usr --with-ssl=openssl --with-sqlite --with-mysql --with-ldap --with-zlib > /dev/null 2> /dev/null

if [ $? != 0 ]; then
  echo "Failed to run configure for Dovecot 2.0"
  exit 1
fi

make > /dev/null 2> /dev/null
if [ $? != 0 ]; then
  echo "Failed to run make for Dovecot 2.0"
  exit 1
fi

if [ ! -e $DIR/dovecot-2.0-pigeonhole ]; then
  echo "Dovecot Pigeonhole is not installed, Downloading a fresh copy of Dovecot Pigeonhole"
  cd $DIR
  hg clone http://hg.rename-it.nl/dovecot-2.0-pigeonhole/ > /dev/null
  if [ $? != 0 ]; then
    echo "Failed to download a fresh copy of Dovecot Pigeonhole"
    exit 1
  fi
  cd $DIR/dovecot-2.0-pigeonhole
else
  cd $DIR/dovecot-2.0-pigeonhole
  hg pull -u > /dev/null
  if [ $? != 0 ]; then
    echo "Failed to update the Dovecot Pigeonhole install"
    exit 1
  fi
  rm -rf *
  hg revert --all > /dev/null
  if [ $? != 0 ]; then
    echo "Failed to restore Dovecot Pigeonhole install"
    exit 1
  fi
fi

./autogen.sh > /dev/null 2> /dev/null
if [ $? != 0 ]; then
  echo "Failed to run autogen.sh for Sieve"
  exit 1
fi

./configure --prefix=/usr --with-dovecot=../dovecot-2.0 > /dev/null
if [ $? != 0 ]; then
  echo "Failed to run configure for Sieve"
  exit 1
fi

make > /dev/null 2> /dev/null
if [ $? != 0 ]; then
  echo "Failed to run make for Sieve"
  exit 1
fi

echo "Stopping Dovecot"
service dovecot stop
if [ $? != 0 ]; then
  echo "Failed to stop the running Dovecot proccess"
fi

cd /var/src/dovecot-2.0
make install > /dev/null 2> /dev/null
if [ $? != 0 ]; then
  echo "failed to install Dovecot"
  exit 1
fi

cd /var/src/dovecot-2.0-pigeonhole
make install > /dev/null 2> /dev/null
if [ $? != 0 ]; then
  echo "Failed to install Sieve"
  exit 1
fi

rm -rf /usr/local/etc/dovecot
ln -s /etc/dovecot /usr/local/etc/dovecot
rm -rf /usr/etc/dovecot
ln -s /etc/dovecot /usr/etc/dovecot

echo "Restarting Dovecot"
service dovecot start

Installing the GIT Daemon for Read Only Access to Repoistory

The idea here is to allow anyone to download your GIT repository using the native git protocol.  This is similar to the post I wrote about Creating a secure Git repository server, but the below method allows full read only public access to the repositories.

Start out by modifying the new file git-daemon.

vim /etc/xinetd.d/git-daemon

Add the below text to the file.

# description: The git daemon offers anonymous access to git repositories
service git
{
    disable        = no
    type           = UNLISTED
    port           = 9418
    socket_type    = stream
    wait           = no
    user           = git
    server         = /usr/libexec/git-core/git-daemon
    server_args    = --inetd --export-all --base-path=/var/git
    log_on_failure += USERID
}

You may need to change the location of the git-daemon (the above example is from Fedora) and you will need to update the location of your git repository directory.

The above configuration will share ALL the git repositories in the /var/git directory.

If you do not wish for the all repositories to be public, you may remove the --export-all flag and add a empty file named git-daemon-export-ok to the git repository you wish to still share.

Enable SSH Key Authorization on FreeNAS

FreeNAS is a powerful tools for archive data and other long-term storage requirements. Recently I have started backing up this and other off-site servers to one of my local FreeNAS boxes. Since these systems are only connected via the insecure internet (no VPN), I decided to transmit the backup files via SSH using SCP.  In order to do this without having to enter my password in for each and every backup (most of which happen while I’m hopeful sleeping), I needed to implement SSH Key Authorization on the receiving FreeNAS box.

To do this first I needed to create a DSA key pair on a different system.  On my Fedora 12 laptop I ran

ssh-keygen -t dsa

The trick here is to not create your new key pair in the default directory of “~/.ssh/” but in a temporary directory instead. So when it asks you.

Generating public/private dsa key pair.
Enter file in which to save the key (/root/.ssh/id_dsa):

Enter a different file in witch to save the key in. Note this is asking for the name of a file, not the name of the directory, it will also create a .pub file, this is the public key for the above private key.

Now that you have a public/private key for the FreeNAS box, if you don’t already you need to create one for the user that you plan on sending the file from.  Just follow the above command, but this time, you may just hit enter all the way threw leaving all options as default.

Next, go to your FreeNAS web-page control panel, If you don’t already have one, you will need to create a user on your FreeNAS box for you to connect to.  You may do this via “Access -> User and Groups” from the black bar on the top of the page.

Now from the top (black) bar go to “Services -> SSH”

On your Services|SSH page, first make sure the service is enabled (top right hand corner). Once it is you will be able to change the below options.

  • TCP Port: The default (port 22) is fine in most cases, but note if you change it, you must use the new port for all connections.
  • Permit root login: My option is that the root account should never be allowed to log in via a remote process.  You should set your system up correctly where this is not needed.
  • Password authentication: For now this must be enabled (checked), once you have set “Key Authorization” up, you may disable this option.
  • TCP forwarding: Disabled (unchecked)
  • Compression: Enabled (checked)
  • Private Key: This will be were we put the private key created above.  All you need to do is copy and paste.
  • Extra options: Blank

After you have made your changes, “Save and Restart” the service.

On your local system, you need to copy the content of the local users public (~/.ssh/username.pub) key file to a new file named “authorized_keys” (Note: this is not the file we created for FreeNAS but the file we created for your local account).  This is the file that will need to be copied to your FreeNAS box.

Now that you have all the needed bits, we need to log into your FreeNAS server and create a “.ssh” directory to store the “authorized_keys” file.  To log in to the FreeNAS box interactively run a command similar too.

ssh freenasuser@freenasaddress

Or if you changed the “TCP Port” above, your command will look like this:

ssh -P freenasport freenasuser@freenasaddress

Once your logged in, you need to create the directory, by doing.

mkdir ~/.ssh

After you have successfully created your directory, you may exit out of your FreeNAS box for the next step.
Back on your local system you need to copy the “authorized_keys” file created before to the FreeNAS box. Using SCP you can do this by running a command like:

scp -P freenasport authorized_keys freenasuser@freenasaddress:~/.ssh/

This will copy the file to the FreeNAS box. Next, reconnect to the FreeNAS box as you did before and run.

cd ~/.ssh/
chmod 600 authorized_keys

Once your done, you should be able to connect to your FreeNAS box using the private key in the authorized_keys with out a password.

Setting the Time Zone on Fedora from the shell

By default Fedora set’s the timezone to UTC, during the install you may change this. But, if you move a server or forget to set the timezone during install.  You may change it after the fact.

Lately I  moved a system from the EST to CST (From Virginia to Minneapolis).  Since this was a webserver, there is no GUI installed (wasting memory).  So I connected to the system via SSH and ran the following command:

cp /usr/share/zoneinfo/America/Chicago /etc/localtime

All your really doing is copy the timezone info for Chicago, into the local systems clock.

To get a list of your time zone, on Fedora, go to your “/usr/share/zoneinfo” directory and list the contents.

cd /usr/share/zoneinfo/
ls

This will give you a list of all the top level regions, find yours and go into it.

cd America

Now find the city that’s closest to you and in your same timezone.

So if your in the Central time zone, copy the Chicago file into your “/etc/localtime” file

cp Chicago /etc/localtime

now check to make sure it all worked.

$ date
Sun Aug 23 15:10:03 CDT 2009

Since this system is in Minneapolis, and Minneapolis is in the CST/CDT I’m good to go.