Alan's Ramblings - Tech category

Why I'm ditching the Arduino software platform

Wed, 29 Feb 2012 20:50:00 GMT

I'm getting set up for my next project and decided to update my development environment. I've finally decided to entirely ditch the Arduino software environment and just use the boards. I stopped using the Arduino IDE some time ago, but now I'm going whole hog and ditching the Arduino library as well. Why? Well, it's simple:

Significant parts of it are pile of junk.

I know that's a pretty strong statement, so I better back it up with evidence. OK, let's start with the hardware serial IO code. Before version 1.0 of the Arduino platform, although reading from the serial ports was interrupt-driven, writing wasn't. Rather, the code went into a spin loop, polling the transmit status bit until the USART was idle before sending the next character. Why was that a problem? Well if you wrote a 80-character string at 9600 baud it would take (8 bits + 1 start bit + 1 stop bit) * 80 / 9600 = 0.083, i.e. 83 milliseconds. That's a huge amount of time for the CPU to be spending just to do some output. I found a number of posts where people were complaining that doing reasonable amounts of IO screwed up all the other bits of their sketches, and no wonder. Admittedly the Arduino 1.0 release notes say that's been changed so that output now uses interrupts as well, but that's not the end of the problems.

Let's take a peek at the HardwareSerial.cpp class. First thing to note is that two 64-byte buffers are allocated for each USART, even if it isn't used. That's 128 bytes on a Duemilanove and 512 bytes on a Mega, or 6% and 12% of the available SRAM respectively. On the Duemilanove that's reasonable as there's only 1 UART, but on the Mega it represents a significant waste of precious memory when only 1 USART is normally going to be in use.

OK, let's look at the new write() function that does interrupt-driven output:

size_t HardwareSerial::write(uint8_t c)
{
  int i = (_tx_buffer->head + 1) % SERIAL_BUFFER_SIZE;

  // If the output buffer is full, there's nothing for it other than to
  // wait for the interrupt handler to empty it a bit
  // ???: return 0 here instead?
  while (i == _tx_buffer->tail)
    ;

  _tx_buffer->buffer[_tx_buffer->head] = c;
  _tx_buffer->head = i;

  sbi(*_ucsrb, _udrie);
  
  return 1;
}

Is there a problem? Let's look at the definition of _tx_buffer:

struct ring_buffer
{
  unsigned char buffer[SERIAL_BUFFER_SIZE];
  volatile unsigned int head;
  volatile unsigned int tail;
};

Oh dear. head and tail are declared as int, i.e. 16 bits, 2 bytes. They are accessed by both the write routine and the interrupt service routine that actually transmits the data yet there's no locking in the write routine so the accesses aren't atomic. Why is that an issue? Well, the avr-libc documentation makes it clear:

A typical example that requires atomic access is a 16 (or more) bit variable that is shared between the main execution path and an ISR. While declaring such a variable as volatile ensures that the compiler will not optimize accesses to it away, it does not guarantee atomic access to it.

The documentation goes on to explain the sorts of symptoms you'll see if you ignore this, follow the link above if you want the full details. This is inexcusably shoddy code - the constraints on accessing variables that are shared between ISR and non-ISR code are well-known. What really concerns me is that people will use the Arduino code as an example of 'good' AVR code and it isn't, in many places it's frankly awful.

"So what?" you say, "That's only one chunk of code that's a bit naff." Unfortunately it's not an isolated instance. Let's move on now to look at one of the newer features that has been added to the Arduino platform, the re-implemented String class. Ok, let's build a minimal program that uses it:

#include "WString.h"
int main(void) {
    String bloat = "hello world";
    return 0;
}

And let's build it:

WString.cpp: In member function ‘int String::lastIndexOf(char, unsigned int) const’:
WString.cpp:503:38: error: comparison of unsigned expression < 0 is always false [-Werror=type-limits]
WString.cpp: In member function ‘int String::lastIndexOf(const String&, unsigned int) const’:
WString.cpp:519:63: error: comparison of unsigned expression < 0 is always false [-Werror=type-limits]

Sigh. One would think that the Arduino developers would at least turn on warnings when they are compiling their code, but they don't. And in this case, the consequence is a bug. So, temporarily comment out the offending lines so we get a successful build, and:

/opt/avr-gcc/bin/avr-size build/test.elf
   text    data     bss     dec     hex filename
  10194      20       5   10219    27eb build/test.elf

Can that really be right? 10K for a one-line program? Unfortunately it is. Any mention of String pulls in the entirety of the class, as well as all the other avr-libc routines it references. So on a Duemilanove that only has 32k to start with, a third of the available memory is gone before you start. At the time the class was being rewritten I expressed my opinion that it was probably a bad idea and that the Arduino developers really needed to target the platform they actually had and not the one they wished they had. And that's not the end of the issues with the String class - on a constrained-memory platform such as the AVR, providing a class like String that relies on malloc, creates lots of temporaries, fragments the (tiny) heap and has no real ability to deal with out-of-memory conditions is a recipe for problems, problems that will manifest themselves as random, mysterious and un-diagnosable run-time errors. And sure enough, a quick google shows that's exactly what tends to happen - just about the worst possible outcome for a platform that's targeted at neophytes.

That's just two examples - there are others as well, such as the well-known performance problems with pin access, which may be up to 50x slower that direct pin access. In fact the only two remaining parts of the Arduino libraries that I still use are the millisecond clock and the serial IO, and they are easy enough to replace, so that's what I'm doing.

While I applaud the aims of the Arduino project, the realities of the restricted hardware platform have to be taken into consideration. In addition, one of the aims of the project is to:

provide a well-designed, maintainable, and stable platform for the future

and despite its unquestionable success on many other fronts, on that one I feel the Arduino platform is less than entirely successful. I for one won't be using any of the software any more, it's just not what I consider to be acceptable quality.

Scala snippet of the day

Fri, 17 Feb 2012 14:04:00 GMT

The following is useful for benchmarking code from inside the REPL:

def timed(op: => Unit) = {
  val start = System.nanoTime
  op
  (System.nanoTime - start) / 1e9
}

Then you can use it like this:

scala> timed { Thread.sleep(5000) }
res0: Double = 5.010083526

That's syntactic sugar for an anonymous function definition and a call of timed. If a function takes only a single parameter you can pass it inside { }, so if we wrote that out longhand it would be:

scala> def fn = Thread.sleep(5000)
fn: Unit

scala> timed(fn)
res0: Double = 5.010088248

The other neat bit is the use of Scala's pass-by-name mechanism to pass the op parameter. That's the funny-looking op: => Unit argument list to timed. Normally Scala uses by-value parameters, i.e. the line timed(fn) would result in the evaluation of fn, and then passing the returned value (in this case, nothing) into timed. However by using pass-by-name, the evaluation of fn takes place inside timed, and by wrapping it inside timing instrumentation we can time how long op takes to execute.

Stupid BT router...

Wed, 23 Nov 2011 14:12:00 GMT

I have the misfortune to own a BT Business 2Wire router, and bleaklow.com sits behind that. A while back it threw a fit and reconfigured itself and when I beat it into submission I didn't notice that it had also lost the port mappings for bleaklow.com. As a result I've been off-air for quite some time. Bloody BT...

Hints on creating 256-colour themes for vim

Wed, 21 Sep 2011 20:06:00 GMT

Most existing vim colour themes are designed for the graphical version of vim, gvim. They don't work properly at all if you are running vim in a terminal, and there are very few themes available for vim in terminal-mode. You are pretty much obliged to make your own if one of the small number of available ones doesn't suit. Here's a list of hints and tips that I discovered in the process of creating one:

If your shell's setting for $TERM is xterm you won't get colour support. It needs to be xtermc or you need to add a set term=xtermc in your .vimrc.
Even with the terminal type set you may not get 256 colours if you are using gnome-terminal. To fix that, put set t_Co=256 in your .vimrc.
In vim, type :help highlight to get instructions on how to specify colours in vim.
To make the process of authoring a colour theme easier, install the Mkcolorscheme.vim script in your vim plugin directory, uncommenting the four commands at the top of the file. Read the instructions in the plugin, they give you hints on how to identify the colours currently being used by syntax elements.
Open a new subwindow in vim and type :so $VIMRUNTIME/syntax/hitest.vim. That will display a table of the current settings, which will be updated as you interactively change colours.
Open this colour chart in your browser, it gives you the colour numbers you'll need to set the colours.
Open up a source file of the type that you want to set the syntax colouring for, put the cursor over an element and use the :GetSyntax command to identify the element type, followed by the appropriate:highlight command to set the element colour.
When you are done, use the :Mkcolorscheme command to generate a set of commands to generate the colour scheme settings. Delete any lines containing cleared as they aren't valid.
Save the colour scheme to a file in your vim colors directory and load the scheme with a colorscheme command in your .vimrc.

Beautiful Scala

Tue, 23 Aug 2011 08:24:23 GMT

This is, of course, beautiful:

def msort[T](less: (T, T) => Boolean)
      (xs: List[T]): List[T] = {
  
    def merge(xs: List[T], ys: List[T]): List[T] =
      (xs, ys) match {
        case (Nil, _) => ys
        case (_, Nil) => xs
        case (x :: xs1, y :: ys1) =>
          if (less(x, y)) x :: merge(xs1, ys)
          else y :: merge(xs, ys1)
      }
  
    val n = xs.length / 2
    if (n == 0) xs
    else {
      val (ys, zs) = xs splitAt n
      merge(msort(less)(ys), msort(less)(zs))
    }
  }

scala> val intSort = msort((x: Int, y: Int) => x < y) _
intSort: (List[Int]) => List[Int] = <function1>
scala> intSort(List(3, 5, 1, 6, 2))
res3: List[Int] = List(1, 2, 3, 5, 6)

This is an example of a recursive merge sort implemented using a technique known as currying, expressed in the language Scala.

Some context: I was looking for a Java-based scripting language to act as 'glue' for my Java interface to OziExplorer which I'm using for a fenceline survey I'm doing for the Moors For The Future project. I initially looked at Groovy and came across an intriguing comment by James Strachan, the creator of Groovy:

I can honestly say if someone had shown me the Programming in Scala book by by Martin Odersky, Lex Spoon & Bill Venners back in 2003 I'd probably have never created Groovy.

That set me off on an exploration of the Scala website which has lots of good tutorials and links, including one to an online interpreter you can type examples into. Scala is a really intriguing language - it sits on top of the JVM so you get 'for free' Java's cross-platform support and access to the huge Java ecosystem, but Scala also blends together the features of some of the most influential languages that preceded it:

The 'everything is an object' nature of Smalltalk or Ruby
The imperative nature of languages that have C or Java heritage
Functional programming as per ML or Haskell
Strongly typed but with sophisticated type inferencing that makes it look & feel similar to Python or Groovy
Uniform access to methods and fields, like Eiffel
Actor-based concurrency similar to Erlang

Scala also adds new features of its own as well, such as abstract types, traits and extractors. It's also very easy to write domain-specific languages in Scala. James Strachan wrote a good summary of Scala, from which the above quote is taken.

I also ordered a copy of Programming in Scala, Second Edition which I'm enjoying reading and can thoroughly recommend. Unlike many programming books it's well written with a light but not overly-chatty style. It explicitly assumes you have programming experience, so it picks up pace pretty quickly. As I've been going through the book, most of my "Why did they do that?" questions have had good answers, which gives the impression that Scala has been very carefully thought through - always a good sign.

Now all I have to do is to think of a substantial project I can use Scala on :-)