Fun and games with Apache, mod_perl, TWiki and SMF, Part 1

As part of the day job I've had to set up a single machine hosting several SSL-secured websites that make heavy use of mod_perl and TWiki in particular. The sites all access some shared content (e.g. source code trees) as well as having their own individual stuff, and they are all laid out in broadly the same way. Another requirement is to be able to easily and quickly add more sites in the future. Being a lazy sort of person, I didn't particularly fancy doing that all by hand, so I dug out my copy of Writing Apache Modules with Perl and C and started hacking. One of the little-known features of mod_perl is that as well as being able to use it to speed up your CGI scripts you can also use it to dynamically create the contents of your httpd.conf file. In my case, some of the configuration (e.g. icons, stylesheets and so on) was common to all the sites I was building, so it could go in httpd.conf as normal. The rest would be done dynamically, so I split that off into a separate file (more on that in a later post) and loaded in via the following block in httpd.conf:

PerlFreshRestart        on
PerlTaintCheck          on
PerlRequire             conf/perl.conf

One of the other things I wanted to do was to be able to use the same CGI scripts running under mod_perl's Apache::Registry on each of the virtual hosts, so I went one step further and abstracted out configuration which was common to both the webserver and the CGI scripts into a seperate file. This file has a hash containing an entry for each virtual host, as well as hashes describing content shared between the various virtual hosts. I'm not going to reproduce the whole thing as it is rather large, but the following excerpts will give a flavour:

# Common configuration items.
our %CommonConf = (
    src                 => '/approot/src',
    twiki               => '/approot/apache/twiki',
    site_stylesheet     => '/stylesheets/site.css',
    src_stylesheet      => '/stylesheets/sccs.css',
    cgi_stylesheet      => '/stylesheets/cgi.css',
);

our %GateConf = (
    # ON & friends.
    'ONNV (all)' => {
        dir     => 'onnv',
        path    => 'usr/src',
        include => [ 'usr/src/include' ],
        desc    => q{
            Entire ON Nevada source hierarchy.
        },
    },
    'ONNV (uts)' => {
        dir     => 'onnv',
        path    => 'usr/src/uts',
        include => [ 'usr/src/include' ],
        desc    => q{
            Kernel-only ON Nevada source hierarchy.
        },
    },
];

our %SiteConf = (
    sitea => {
        interface       => 'bge0:1',
        wiki_main       => 'Sitea',
        wiki_webs       => [ 'Sitea' ],
        wiki_search     => '/twiki/bin/search/Sitea/SearchResult?search=',
        wiki_view       => '/twiki/bin/view/Sitea',
        gates   => [
            'ONNV (all)',
            'ONNV (uts)',
        ],
    },
);

The first things of interest are the keys of the %SiteConf hash, which are the names of the virtual hosts I was going to create, and the the interface value in %SiteConf which is the virtual interface I was going to run the host on. There's a chicken-and-egg situation when you try to set up SSL-enabled virtual hosts in Apache. Normal Apache name-based virtual hosting uses the hostname in the incoming HTTP requests to figure out which virtual host you are accessing. Unfortunately when running SSL that isn't possible because the incoming request is encrypted - so Apache can't find out whick decryption key to use because the virtual hostname is encrypted! The consequence is that you need to assign each SSL virtual host it's own IP address, and therefore it's own (virtual) interface. The interface value is used by the SMF service script that is used to start up the webserver - this wraps around the standard apachectl script and creates/deletes the required virtual interfaces as necessary before calling the standard apachectl script:

#!/bin/ksh -p
#
# SMF service script for Apache.
#

# Include SMF shell support.
. /lib/svc/share/smf_include.sh

# Root of application install tree.
typeset -r APP_ROOT=${APP_ROOT:-$(APP_ROOT)}

# Apache configuration
typeset -r APACHE_HOME=${APP_ROOT}/apache
typeset -r APACHE_LOG=$APACHE_HOME/logs
typeset -r APACHE_BIN=$APACHE_HOME/bin
typeset -r NETMASK=255.255.255.0

if [[ $# -ne 1 ]]; then
        echo "Invalid arguments"
        exit $SMF_EXIT_ERR_FATAL
fi

# Clear SMV environment variables.
smf_clear_env

case $1 in
start)
        /bin/rm -rf $APACHE_LOG/*
        for vhost in $(perl -I$APP_ROOT/conf -MAppConf \
            -e 'AppConf::print_hosts()'); do
                if=$(perl -I$APP_ROOT/conf -MAppConf \
                    -e "AppConf::print_interface('$vhost')")
                /usr/sbin/ifconfig $if plumb up $vhost netmask $NETMASK
        done
        $APACHE_BIN/apachectl startssl \
            && exit $SMF_EXIT_OK || exit $SMF_EXIT_ERR_FATAL
        ;;
stop)
        $APACHE_BIN/apachectl stop
        status=$?
        for vhost in $(perl -I$APP_ROOT/conf -MAppConf \
            -e 'AppConf::print_hosts()'); do
                if=$(perl -I$APP_ROOT/conf -MAppConf \
                    -e "AppConf::print_interface('$vhost')")
                /usr/sbin/ifconfig $if unplumb
        done
        [[ $status -eq 0 ]] && exit $SMF_EXIT_OK || exit $SMF_EXIT_ERR_FATAL
        ;;
refresh)
        $APACHE_BIN/apachectl graceful \
            && exit $SMF_EXIT_OK || exit $SMF_EXIT_ERR_FATAL
        ;;
*)
        echo "Invalid method $1"
        exit $SMF_EXIT_ERR_FATAL
        ;;
esac

The AppConf::print_hosts() and AppConf::print_interface() subs are provided by the common configuration file so that shell scripts (such as the SMF service script) can query the list of virtual hosts and interfaces. The other bit of the jigsaw is the SMF manifest that describes the Apache service:

<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">
<!--
    start/stop/restart apache for app
-->
<service_bundle type='manifest' name='app:apache'>
<service name='application/app/apache' type='service' version='1'>
    <single_instance />
    <instance name='default' enabled='true'>
        <!-- milestones required -->
        <dependency name='milestone'
            grouping='require_all'
            restart_on='error'
            type='service'>
            <service_fmri value='svc:/milestone/multi-user-server:default' />
        </dependency>
        <dependency name='filesystem'
            grouping='require_all'
            restart_on='error'
            type='service'>
            <service_fmri value='svc:/system/filesystem/local:default' />
        </dependency>
        <!-- services required -->
        <dependency name='database'
            grouping='require_all'
            restart_on='refresh'
            type='service'>
            <service_fmri value='svc:/application/app/mysql:default' />
        </dependency>
        <!-- default method context -->
        <method_context working_directory=':default' project=':default'>
            <method_credential user='root' group='root' />
            <method_environment>
                <envvar name='PATH'
                value='/approot/apache/bin:/aproot/utils/bin:/approot/perl5/bin:/usr/bin' />
            </method_environment>
        </method_context>
        <!-- methods -->
        <exec_method
            type='method'
            name='start'
            exec='apache start'
            timeout_seconds='60'
        />
        <exec_method
            type='method'
            name='stop'
            exec='apache stop'
            timeout_seconds='60'
        />
        <exec_method
            type='method'
            name='refresh'
            exec='apache refresh'
            timeout_seconds='60'
        />
    </instance>
    <stability value='Stable' />
    <template>
        <common_name>
            <loctext xml:lang='C'>
            app application - apache component
            </loctext>
        </common_name>
        <documentation>
            <doc_link name='apache'
              uri='file:////approot/apache/htdocs/manual' />
        </documentation>
    </template>
</service>
</service_bundle>

This is a pretty standard SMF script, things to note are the dependency on another component of the application (the MySQL service) and the use of the little-used method_context element to factor out the common environment (UID, working directory and PATH) from the individual start/stop/restart method definitions.

In part 2 I'll describe how I scripted the configuration of the virtual hosts, including putting them under the control of the Fair Share scheduler from within perl.

Categories : Web, Tech, Perl, Work