Drystone walling

"All Derbyshire is full of steep hills and you see neither hedge nor tree but only low drye stone walls"
Celia Fiennes - 1697

Today I got a chance to try my hand at something I've been wanting to have a go at for a long time - drystone walling. For those of you who don't know what this is, it's a technique for building walls using just stone and with absolutely no mortar, hence the name. In Derbyshire the practice dates back to at least 2500 BC, with systematic building starting in the 14th and 15th centuries and carrying all the way through to the 19th century. The Parliamentary Enclosure Act of 1750 saw the building of standardised walls, paid for the landowners wo were claiming the land for their own use. By 1820 most of the existing walls were complete, and since then there has been a gradual decline, with many of the walls falling into disrepair. More recently there has been a resurgence in interest in preserving and maintaining the walls, which form a distinctive feature of the Peak District landscape. The number of professional wallers is beginning to climb both in response to this interest and also because of the availability of government grants for walling work.

The section we were working on was an existing wall that had become derelict, alongside the Pennine Bridleway at Hayfield. The first job was to take down the remnants of the existing wall, laying the stone to one side so we could reuse it. The foundations were OK, so we didn't need to dig right down - when starting a fresh wall the top 6 inches of soil need to be dug out to give a firm footing. Most field walls are 4'6" high and are 2' wide at the base. As the wall rises it gets narrower, the amount is known as the 'batter' of the wall, in our case we were using a ratio of 1:8, so at the top our wall needed to be about a foot wide - we used metal bars at either end of the section with strings between them as a guide, moving the strings up the bars as the wall rose.

Drystone walling

The wall is actually made up of several different components. Free-standing walls are double-skinned with each face being built seperately, so they can be thought of almost as two walls leaning against each other. The components of the wall are as follows:

  • Footings. These are the largest (and heaviest!) stones and take the entire weight of the wall - there can be between 1 and 2 tons of stone per meter of wall - and my back says that's probably about right!
  • Face stones. These make up the two outside faces of the wall, and start off with the largest ones at the bottom. Narrow stones are placed with their long edges into the wall, not along it in order to give the wall strength.
  • Hearting. This is small stone used to pack the middle of the wall between the faces. Soil isn't used as it would wash out over time - a well-built wall will last for over 100 years.
  • Throughs. These span the entire width of the wall and tie the two faces together. On a 4'6" wall they are about 2' from the ground, spaced about a yard apart along the wall.
  • Copestones. Usually semicircular, these fit vertically on top of the wall and again tie the two faces of the wall together.

Drystone walling

Here you can see the base of the wall, showing the two faces and the heartings between them.

Drystone walling

The wall is a bit higher here and you can see that it is starting to narrow. Although the stone is all irregular, it is considered to be cheating to dress it to shape. The aim is to build in regular horizontal courses (as far as possible), and to overlap the vertical joints in the same way as is done for brick walls.

Drystone walling

Here the wall is at full height, with the top being levelled up with small face stones so that the copestones will sit evenly across the wall.

Drystone walling

The final stage is to add the vertical copestones that tie the top of the wall together.. Some walls have dressed semicircular copestones of a regular size, but here we are going for the rustic look - actually, we didn't have any dressed stone!

Job done, and I have the aching back and bruised fingernail to prove it! As I said I'd always wanted to have a go at walling, and I was lucky enough to have a fine bright autumn day (and pleasant company to boot!) for my first attempt. Many thanks to Terry for showing me the tricks of the trade, and I can't wait to go back to do a bit more. Hopefully the wall I helped to build today will last considerably longer than I will :-)

Alan in wonDLLand

Yesterday evening I had another poke at the Java wrapper for OziExplorer that I'm sporadically working on, OziExplorer is the GPS mapping application that I use. It's a pretty fat API so I'm gradually chipping away at it. My latest tweak was to add a simple Swing GUI to my test app, but as soon as I did it blew up every time I exited with two rather foreboding messages, The exception unknown software exception (oxoeedfade) occurred in the application at location 0x7c81eb33 and Runtime error 217 at 0009D54. Google told me that second message was coming from Delphi - OziAPI.dll is generated with Delphi, but didn't shed much light on exactly what was causing the error. Some experimentation revealed that the problem was actually nothing to do with Swing, I could get the same effect by just calling System.exit immediately after calling any of the functions in the Delphi DLL. If I just let the program fall off the end of main the problem didn't happen, which suggested that whatever was going belly up was being caused by something that System.exit was doing, either directly or by indirectly perturbing the ordering of the JVM exit processing.

A bit more googling revealed that other people had hit the same problem and that one workaround was to manually open the DLL with LoadLibrary, find the symbol addresses via GetProcAddress and finally manually free the DLL with FreeLibrary on exit. (For those in the Solaris seats that's the Win32 equivalent of dlopen, dlysm and dlclose). This suggested to me that the problem was something to do with the timing of the DLL initialisation and/or cleanup. In my case I was pulling in the OziAPI DLL via a linker dependency from the JNI DLL, and I was loathe to have to manually pull in OziAPI.dll and look up the symbols just to work around this problem. Some more spelunking through the Microsoft documentation revealed the following interesting snippets:

If you terminate a process by calling TerminateProcess or TerminateJobObject, the DLLs of that process do not receive DLL_PROCESS_DETACH notifications. If you terminate a thread by calling TerminateThread, the DLLs of that thread do not receive DLL_THREAD_DETACH notifications.

The TerminateProcess function is used to unconditionally cause a process to exit. The state of global data maintained by dynamic-link libraries (DLLs) may be compromised if TerminateProcess is used rather than ExitProcess.

I grabbed a copy of strace for NT and traced the execution of my test application, and although I could see calls to TerminateProcess I couldn't see any calls at all to ExitProcess, not only when I called System.exit but also when I let the program fall of the end of main. Hmm. If that was what was happening there didn't seem to be much I could do to change the behaviour of System.exit, but I already knew that if the process exited by falling off the end of main the problem didn't occur - what I needed to do was to find out how to make that happen from inside a Swing callback. I was wandering around the JDK documentation when I found an interesting document titled AWT Threading Issues, which says:

Therefore, a stand-alone AWT application that wishes to exit cleanly without calling System.exit must:
  • Make sure that all AWT or Swing components are made undisplayable when the application finishes. This can be done by calling Window.dispose on all top-level Windows. See Frame.getFrames.
  • Make sure that no method of AWT event listeners registered by the application with any AWT or Swing component can run into an infinite loop or hang indefinitely. For example, an AWT listener method triggered by some AWT event can post a new AWT event of the same type to the EventQueue. The argument is that methods of AWT event listeners are typically executed on helper threads.

So the code to work around my problem turned out to be trivial, I just needed to put the following in my exit button callback to make sure that all the threads in the application exit cleanly:

        for (Frame f : Frame.getFrames()) {
            f.dispose();
        }

Hey presto, everything now works. Obviously if the application creates it's own threads they will also need to be gracefully terminated. I'm still discussing the exact cause of the problem with the JVM folks, but for now I'm happy that I've got a workaround. Hopefully this information will save someone some grief as my searching revealed that quite a few people had been hit by this issue but that nobody quite understood how to work around it other than the nasty manual DLL loading hack.

Categories : Java, Tech

DTrace and Perl

After reading Bryan's DTrace and PHP and DTrace and PHP, demonstrated blog entries, I thought I'd better have a crack at doing the same for Perl - after all, anything PHP can do, Perl can do better - right? ;-)

I posted a question on the perl5-porters list about the best way of doing this for Perl, and Dave Mitchell kindly stepped up to help - many thanks to Dave, and he should take most of the credit for figuring out how best to do this.

The PHP DTrace probes instrument the function entry and return points within a PHP script so I thought I'd do the same for Perl. After some to-and-fro-ing with Dave it became apparent that the easiest way was to add DTrace probes to the PUSHSUB and POPSUB macros. As the names suggest, these are called each time a Perl subroutine is entered and exited. This doesn't get all of the potential entry points, for example eval and require but it's a good start, at least for the purposes of prototyping. The first step is to define the DTrace probes that we will be using:

provider perl {
        probe sub__entry(char*, char*, int);
        probe sub__return(char*, char*, int);
};

The arguments are the sub name, the filename and line number. Here's the changes that are required to cop.h for the PUSHSUB case:

        DTRACE_PROBE3(perl, sub__entry, GvENAME(CvGV(cv)),              \
            CopFILE((COP*)CvSTART(cv)), CopLINE((COP*)CvSTART(cv)));

The fiddly bits are the dereferencing of the current cv to extract the sub name, file name and line number. The next bit is to hack Makefile to call dtrace to postprocess the object files before building libperl.so. This builds an extra object file which we then need to include in the file list used to build libperl.so:

$(LIBPERL): $& perl$(OBJ_EXT) $(obj) $(LIBPERLEXPORT)
        dtrace -G -s dtrace.d -o dtrace.o $(obj) miniperlmain.o perl.o
        $(LD) -o $@ $(SHRPLDFLAGS) perl$(OBJ_EXT) $(obj) dtrace.o $(libs)

Job done. There are a couple of caveats however - firstly, adding the probes has a worst-case performance hit of about 5% even when they aren't enabled. The hit depends on what you are doing in your script - the worst case is for a script that does nothing but call an empty sub. The DTrace probes themselves are NOP'd out when they are inactive, but the code used to calculate the sub name, file name and line number and put them on the stack is still executed even when DTrace is inactive. I've discussed this with the DTrace crew and a way to minimise the overhead is to use a DTrace translator - a shim that sits between the application and the DTrace script and gives a stable view of application data strucures. This would take a pointer to the current cv and do the various dereferencing ops to get the sub name, file name and line number only when the probe was actually active rather than all the time, as in the prototype. This is a good idea not just because of the performance improvement, but also because it means that we can provide a stable API that will be immune to changes in the perl interpreter.

The second caveat is that at the moment some tests in the perl test suite fail with the DTrace macros inserted. Most of these happen because the object files that go into some of the extensions aren't postprocessed with DTrace, so if the extension uses the modified PUSHSUB and POPSUB macros it will fail. Two solutions spring to mind - either modify MakeMaker so that module objects are postprocessed with DTrace, or conditionally add the DTrace probes only when compiling libperl. Overall, I think I prefer the second option, at least as a first pass option.

Anyway, without further ado I present a simple example of it in use. The following D script aggregates page faults by the sub that generated them. Page faults that are caused by the perl interpreter when it isn't actually executing perl code (e.g. during startup) are aggregated under PERL

#pragma D option quiet

BEGIN
{
        self->sub = "PERL";
        self->file = "PERL";
}

perl$target:::sub-entry
{
        self->sub = copyinstr(arg0);
        self->file = copyinstr(arg1);
}

perl$target:::sub-return
{
        self->sub = "PERL";
        self->file = "PERL";
}

vminfo:::maj_fault,
vminfo:::zfod,
vminfo:::as_fault
/self->sub != 0/
{
        @pf[self->sub, self->file] = count();
}

END
{
        printa("%@8d  %-20s %s\n", @pf);
}

And here is the output when run on pod2text perl.pod:

       1  import               /demo/lib/Exporter.pm
       1  import               /demo/lib/strict.pm
       1  import               /demo/lib/warnings/register.pm
       1  output               /demo/lib/Pod/Text.pm
       2  GetOptions           /demo/lib/Getopt/Long.pm
       2  bits                 /demo/lib/strict.pm
       2  interpolate          /demo/lib/Pod/Parser.pm
       2  seq_c                /demo/lib/Pod/Text.pm
       2  wrap                 /demo/lib/Pod/Text.pm
       4  as_heavy             /demo/lib/Exporter.pm
       6  verbatim             /demo/lib/Pod/Text.pm
       8  BEGIN                /demo/lib/Pod/Select.pm
       8  BEGIN                /demo/lib/Pod/Usage.pm
      10  BEGIN                /demo/lib/Pod/Text.pm
      11  import               /demo/lib/vars.pm
      13  BEGIN                /demo/lib/Pod/Parser.pm
      18  BEGIN                /demo/lib/vars.pm
      25  BEGIN                pod/pod2text
      28  BEGIN                /demo/lib/Getopt/Long.pm
      51  BEGIN                /demo/lib/warnings.pm
    1580  PERL                 PERL

Of course that's just a really simplistic sample, the full power of DTrace can be used to find out exactly what your perl script is doing sub-by-sub, whether it be in libperl, libc or the kernel itself. Here's an example which shows which perl subs result in kernel locks being taken, the output is for the same pod2text perl.pod command. If you want to know what all the scary numbers mean, the DTrace lockstat provider documentation explains all.

       1 adaptive BEGIN                          /demo/lib/Symbol.pm
       1 adaptive ConfigDefaults                 /demo/lib/Getopt/Long.pm
       1 adaptive DESTROY                        /demo/lib/Config.pm
       1 adaptive FindOption                     /demo/lib/Getopt/Long.pm
       1 adaptive MAIN                           pod/pod2text
       1 adaptive TIEHASH                        /demo/lib/Config.pm
       1 adaptive _pop_input_stream              /demo/lib/Pod/Parser.pm
       1 adaptive _push_input_stream             /demo/lib/Pod/Parser.pm
       1 adaptive begin_input                    /demo/lib/Pod/Parser.pm
       1 adaptive begin_pod                      /demo/lib/Pod/Parser.pm
       1 adaptive cmd_back                       /demo/lib/Pod/Text.pm
       1 adaptive cmd_over                       /demo/lib/Pod/Text.pm
       1 adaptive config                         /demo/lib/Getopt/Long.pm
       1 adaptive end_input                      /demo/lib/Pod/Parser.pm
       1 adaptive end_pod                        /demo/lib/Pod/Parser.pm
       1 adaptive export                         /demo/lib/Exporter.pm
       1 adaptive import                         /demo/lib/Config.pm
       1 adaptive import                         /demo/lib/Getopt/Long.pm
       1 adaptive initialize                     /demo/lib/Pod/Parser.pm
       1 adaptive initialize                     /demo/lib/Pod/Text.pm
       1 adaptive new                            /demo/lib/Pod/Parser.pm
       1 adaptive parse_from_file                /demo/lib/Pod/Parser.pm
       1 adaptive parse_from_filehandle          /demo/lib/Pod/Parser.pm
       1 adaptive was_cutting                    /demo/lib/Pod/InputObjects.pm
       2 adaptive BEGIN                          /demo/lib/Config.pm
       2 adaptive BEGIN                          /demo/lib/Exporter/Heavy.pm
       2 adaptive BEGIN                          /demo/lib/File/Spec.pm
       2 adaptive Configure                      /demo/lib/Getopt/Long.pm
       2 adaptive gensym                         /demo/lib/Symbol.pm
       2 spin     import                         /demo/lib/vars.pm
       2 spin     interpolate                    /demo/lib/Pod/Parser.pm
       2 spin     seq_c                          /demo/lib/Pod/Text.pm
       2 spin     verbatim                       /demo/lib/Pod/Text.pm
       3 adaptive BEGIN                          /demo/lib/File/Spec/Unix.pm
       3 adaptive BEGIN                          /demo/lib/Pod/InputObjects.pm
       3 adaptive BEGIN                          /demo/lib/Pod/ParseLink.pm
       3 spin     __ANON__                       /demo/lib/Pod/Parser.pm
       3 spin     input_file                     /demo/lib/Pod/Parser.pm
       3 spin     parse_text                     /demo/lib/Pod/Parser.pm
       3 spin     preprocess_line                /demo/lib/Pod/Parser.pm
       3 spin     wrap                           /demo/lib/Pod/Text.pm
       4 adaptive BEGIN                          /demo/lib/constant.pm
       4 adaptive mkMask                         /demo/lib/warnings/register.pm
       4 adaptive unimport                       /demo/lib/strict.pm
       6 spin     new                            /demo/lib/Pod/InputObjects.pm
       7 adaptive cmd_head2                      /demo/lib/Pod/Text.pm
       8 adaptive seq_f                          /demo/lib/Pod/Text.pm
       9 adaptive import                         /demo/lib/constant.pm
       9 spin     as_heavy                       /demo/lib/Exporter.pm
      11 adaptive cmd_head1                      /demo/lib/Pod/Text.pm
      12 adaptive ParseOptionSpec                /demo/lib/Getopt/Long.pm
      12 adaptive cmd_item                       /demo/lib/Pod/Text.pm
      12 adaptive item                           /demo/lib/Pod/Text.pm
      12 spin     BEGIN                          /demo/lib/Pod/Select.pm
      15 adaptive seq_i                          /demo/lib/Pod/Text.pm
      18 adaptive heading                        /demo/lib/Pod/Text.pm
      18 adaptive import                         /demo/lib/warnings/register.pm
      18 spin     BEGIN                          /demo/lib/Pod/Usage.pm
      26 spin     BEGIN                          /demo/lib/Pod/Text.pm
      32 adaptive command                        /demo/lib/Pod/Text.pm
      33 adaptive GetOptions                     /demo/lib/Getopt/Long.pm
      34 adaptive seq_c                          /demo/lib/Pod/Text.pm
      35 adaptive left_delimiter                 /demo/lib/Pod/InputObjects.pm
      38 adaptive seq_b                          /demo/lib/Pod/Text.pm
      39 adaptive _infer_text                    /demo/lib/Pod/ParseLink.pm
      39 adaptive _parse_section                 /demo/lib/Pod/ParseLink.pm
      39 adaptive file_line                      /demo/lib/Pod/InputObjects.pm
      39 adaptive parselink                      /demo/lib/Pod/ParseLink.pm
      39 adaptive seq_l                          /demo/lib/Pod/Text.pm
      42 spin     BEGIN                          /demo/lib/Getopt/Long.pm
      42 spin     BEGIN                          /demo/lib/Pod/Parser.pm
      42 spin     BEGIN                          pod/pod2text
      57 adaptive reformat                       /demo/lib/Pod/Text.pm
      57 adaptive textblock                      /demo/lib/Pod/Text.pm
      92 adaptive wrap                           /demo/lib/Pod/Text.pm
      96 spin     BEGIN                          /demo/lib/vars.pm
      99 adaptive import                         /demo/lib/Exporter.pm
      99 adaptive verbatim                       /demo/lib/Pod/Text.pm
     100 adaptive import                         /demo/lib/strict.pm
     104 adaptive output_handle                  /demo/lib/Pod/Parser.pm
     111 adaptive as_heavy                       /demo/lib/Exporter.pm
     113 adaptive interior_sequence              /demo/lib/Pod/Text.pm
     113 adaptive parse_tree                     /demo/lib/Pod/InputObjects.pm
     114 adaptive __ANON__                       /demo/lib/Pod/Parser.pm
     119 adaptive cutting                        /demo/lib/Pod/Parser.pm
     119 adaptive parse_paragraph                /demo/lib/Pod/Parser.pm
     119 adaptive preprocess_paragraph           /demo/lib/Pod/Text.pm
     120 adaptive output                         /demo/lib/Pod/Text.pm
     126 adaptive top                            /demo/lib/Pod/InputObjects.pm
     127 adaptive input_file                     /demo/lib/Pod/Parser.pm
     127 adaptive parse_text                     /demo/lib/Pod/Parser.pm
     133 spin     BEGIN                          /demo/lib/warnings.pm
     148 adaptive right_delimiter                /demo/lib/Pod/InputObjects.pm
     159 adaptive interpolate                    /demo/lib/Pod/Parser.pm
     162 adaptive bits                           /demo/lib/strict.pm
     167 adaptive BEGIN                          /demo/lib/Pod/Select.pm
     181 adaptive _set_child2parent_links        /demo/lib/Pod/InputObjects.pm
     183 adaptive cmd_name                       /demo/lib/Pod/InputObjects.pm
     183 adaptive nested                         /demo/lib/Pod/InputObjects.pm
     220 adaptive BEGIN                          /demo/lib/Pod/Usage.pm
     239 adaptive children                       /demo/lib/Pod/InputObjects.pm
     275 adaptive BEGIN                          /demo/lib/Pod/Text.pm
     328 adaptive BEGIN                          /demo/lib/Pod/Parser.pm
     352 adaptive DESTROY                        /demo/lib/Pod/InputObjects.pm
     352 adaptive _unset_child2parent_links      /demo/lib/Pod/InputObjects.pm
     437 adaptive preprocess_line                /demo/lib/Pod/Parser.pm
     455 adaptive import                         /demo/lib/vars.pm
     474 adaptive new                            /demo/lib/Pod/InputObjects.pm
     496 adaptive BEGIN                          /demo/lib/vars.pm
     643 adaptive append                         /demo/lib/Pod/InputObjects.pm
     670 adaptive BEGIN                          pod/pod2text
     695 adaptive BEGIN                          /demo/lib/Getopt/Long.pm
    1243 adaptive BEGIN                          /demo/lib/warnings.pm
    2946 spin     PERL                           PERL
   51019 adaptive PERL                           PERL

Categories : Solaris, Tech, Perl