Why I'm ditching the Arduino software platform

I'm getting set up for my next project and decided to update my development environment. I've finally decided to entirely ditch the Arduino software environment and just use the boards. I stopped using the Arduino IDE some time ago, but now I'm going whole hog and ditching the Arduino library as well. Why? Well, it's simple:

Significant parts of it are pile of junk.

I know that's a pretty strong statement, so I better back it up with evidence. OK, let's start with the hardware serial IO code. Before version 1.0 of the Arduino platform, although reading from the serial ports was interrupt-driven, writing wasn't. Rather, the code went into a spin loop, polling the transmit status bit until the USART was idle before sending the next character. Why was that a problem? Well if you wrote a 80-character string at 9600 baud it would take (8 bits + 1 start bit + 1 stop bit) * 80 / 9600 = 0.083, i.e. 83 milliseconds. That's a huge amount of time for the CPU to be spending just to do some output. I found a number of posts where people were complaining that doing reasonable amounts of IO screwed up all the other bits of their sketches, and no wonder. Admittedly the Arduino 1.0 release notes say that's been changed so that output now uses interrupts as well, but that's not the end of the problems.

Let's take a peek at the HardwareSerial.cpp class. First thing to note is that two 64-byte buffers are allocated for each USART, even if it isn't used. That's 128 bytes on a Duemilanove and 512 bytes on a Mega, or 6% and 12% of the available SRAM respectively. On the Duemilanove that's reasonable as there's only 1 UART, but on the Mega it represents a significant waste of precious memory when only 1 USART is normally going to be in use.

OK, let's look at the new write() function that does interrupt-driven output:

size_t HardwareSerial::write(uint8_t c)
{
  int i = (_tx_buffer->head + 1) % SERIAL_BUFFER_SIZE;

  // If the output buffer is full, there's nothing for it other than to
  // wait for the interrupt handler to empty it a bit
  // ???: return 0 here instead?
  while (i == _tx_buffer->tail)
    ;

  _tx_buffer->buffer[_tx_buffer->head] = c;
  _tx_buffer->head = i;

  sbi(*_ucsrb, _udrie);
  
  return 1;
}

Is there a problem? Let's look at the definition of _tx_buffer:

struct ring_buffer
{
  unsigned char buffer[SERIAL_BUFFER_SIZE];
  volatile unsigned int head;
  volatile unsigned int tail;
};

Oh dear. head and tail are declared as int, i.e. 16 bits, 2 bytes. They are accessed by both the write routine and the interrupt service routine that actually transmits the data yet there's no locking in the write routine so the accesses aren't atomic. Why is that an issue? Well, the avr-libc documentation makes it clear:

A typical example that requires atomic access is a 16 (or more) bit variable that is shared between the main execution path and an ISR. While declaring such a variable as volatile ensures that the compiler will not optimize accesses to it away, it does not guarantee atomic access to it.

The documentation goes on to explain the sorts of symptoms you'll see if you ignore this, follow the link above if you want the full details. This is inexcusably shoddy code - the constraints on accessing variables that are shared between ISR and non-ISR code are well-known. What really concerns me is that people will use the Arduino code as an example of 'good' AVR code and it isn't, in many places it's frankly awful.

"So what?" you say, "That's only one chunk of code that's a bit naff." Unfortunately it's not an isolated instance. Let's move on now to look at one of the newer features that has been added to the Arduino platform, the re-implemented String class. Ok, let's build a minimal program that uses it:

#include "WString.h"
int main(void) {
    String bloat = "hello world";
    return 0;
}

And let's build it:

WString.cpp: In member function ‘int String::lastIndexOf(char, unsigned int) const’:
WString.cpp:503:38: error: comparison of unsigned expression < 0 is always false [-Werror=type-limits]
WString.cpp: In member function ‘int String::lastIndexOf(const String&, unsigned int) const’:
WString.cpp:519:63: error: comparison of unsigned expression < 0 is always false [-Werror=type-limits]

Sigh. One would think that the Arduino developers would at least turn on warnings when they are compiling their code, but they don't. And in this case, the consequence is a bug. So, temporarily comment out the offending lines so we get a successful build, and:

/opt/avr-gcc/bin/avr-size build/test.elf
   text    data     bss     dec     hex filename
  10194      20       5   10219    27eb build/test.elf

Can that really be right? 10K for a one-line program? Unfortunately it is. Any mention of String pulls in the entirety of the class, as well as all the other avr-libc routines it references. So on a Duemilanove that only has 32k to start with, a third of the available memory is gone before you start. At the time the class was being rewritten I expressed my opinion that it was probably a bad idea and that the Arduino developers really needed to target the platform they actually had and not the one they wished they had. And that's not the end of the issues with the String class - on a constrained-memory platform such as the AVR, providing a class like String that relies on malloc, creates lots of temporaries, fragments the (tiny) heap and has no real ability to deal with out-of-memory conditions is a recipe for problems, problems that will manifest themselves as random, mysterious and un-diagnosable run-time errors. And sure enough, a quick google shows that's exactly what tends to happen - just about the worst possible outcome for a platform that's targeted at neophytes.

That's just two examples - there are others as well, such as the well-known performance problems with pin access, which may be up to 50x slower that direct pin access. In fact the only two remaining parts of the Arduino libraries that I still use are the millisecond clock and the serial IO, and they are easy enough to replace, so that's what I'm doing.

While I applaud the aims of the Arduino project, the realities of the restricted hardware platform have to be taken into consideration. In addition, one of the aims of the project is to:

provide a well-designed, maintainable, and stable platform for the future
and despite its unquestionable success on many other fronts, on that one I feel the Arduino platform is less than entirely successful. I for one won't be using any of the software any more, it's just not what I consider to be acceptable quality.

Categories : AVR, Tech


Re: Why I'm ditching the Arduino software platform

Hi there. And what is your alternative?

Atmel Studio 6?

 

Thank you.

Re: Why I'm ditching the Arduino software platform

I'm using the same compiler thay the Arduino platform uses, namely avr-gcc, and libc-avr. On top of that I'm using the NetBeans IDE, as described in this post, although I'm fairly sure Eclipse could be used instead, if that's your preference. The other thing I'm using is a variant of the Makefile I posted a while back with the Arduino-specific parts removed. Once I'm happy it's working OK I'll post a copy of it.

Re: Why I'm ditching the Arduino software platform

Have you considered submitting patches to improve the quality of the Arduino library?

How about a fork at GitHub (or similar)? Please do it!

Re: Why I'm ditching the Arduino software platform

Actually I've just replaced all the Arduino stuff.  I now have an accurate millisecond timer that doesn't glitch as well as interrupt-driven, line-buffered serial IO that allows me to use avr-libc stdio functions such as printf() & scanf() and will easily run at 115KBaud whilst there's a load of other software PWM, timed events and other interrupts going on.

Re: Why I'm ditching the Arduino software platform

The "Next project" link is broken - MS have taken it down!

Interesting stuff about the slowness of the IO on the Arduino. I would argue that basic IO was its reason to be, and to have such poor code for what will likely be 90% of the operations on most projects is quite incredible.

Have you looked at the LaunchPad from TI?

Re: Why I'm ditching the Arduino software platform

Thanks for the note about the broken link, I've fixed it. I've heard of the LaunchPad but despite the low cost I think I'll stick with AVRs - the LaunchPad toolchain seems pretty proprietary, doesn't run on my platform (Solaris) and a chip with only 2K of code space is pretty much useless for the projects I'm interested in.

Re: Why I'm ditching the Arduino software platform

I really like the idea of using the Arduino hardware but moving away from all the hand-holding. It would be just awesome if we can start a community of these "advanced arduino" users that supports the hardware but pushes the software stack forward. 

How are you using the UART serial port? you mention that you're doing it with interrupts - can you post a code example for how you're doing the writing? 

Re: Why I'm ditching the Arduino software platform

I think the idea of a community of more advanced users certainly has merit, but that's pretty much what exists already over on avrfreaks.net, if you haven't visited it before it's certainly worth a browse. As for software stack, I've found in practice that there are only a couple of bits of code I reuse regularly - clock/timer management, serial IO and a task manager. Once you get familiar with the AVR features, it's easy enough to use them directly - that's the advantage of moving away from the hand-holding :-)

The USART code I've written uses interrupts for both input and output, does line buffering, line editing, backspace handling etc. It's on my list to do a writeup and publish the code here on my blog.

Re: Why I'm ditching the Arduino software platform

Have you considered using Wiring rather than Arduino? It seems like many of the issues you bought up is fixed in wiring - 8 bit ints for serial port buffers so you don't have to deal with atomic issues, etc. 

Re: Why I'm ditching the Arduino software platform

I took a brief look at Wiring, but it seems to have been largely supplanted by Arduino, the boards are significantly more expensive - and I couldn't even download the source repo as a SVN checkout just hangs.  Thanks for the suggestion, but I think I'll pass ;-)

Re: Why I'm ditching the Arduino software platform

 Oh, I think they're actually quite active. I'm sure you don't need this, but do take a peek at their github page, that's where the activity is going on.https://github.com/WiringProject/Wiring/

 

I was specifically interested in their Serial implementation: https://github.com/WiringProject/Wiring/blob/master/framework/cores/AVR8Bit/WHardwareSerial.h

I think it looks a bit better than the equivalent Arduino code!

Re: Why I'm ditching the Arduino software platform

I agree that it looks better than the Arduino code (not difficult!) but it is a C++ class so you can't easily use it from C code, plus I want to use avr-libc's stdio functions such as printf() and scanf(), so it's not really of much interest to me personally.

Re: Why I'm ditching the Arduino software platform

I agree with another poster, and think that many would appreciate it if you could either send these patches back to the Arduino team, or publish those patches (whether it be here or on GitHub). As much as I would like to take this challenge myself, there is enough wheel re-inventing these days and honestly I am not that experienced in low level programming. Please don't leave us hanging! :)

Re: Why I'm ditching the Arduino software platform

What I have isn't a set of patches to the Arduino environment, it's a small library of routines that I use as a replacement for the bits of the Arduino libraries that I was still actually using - mostly the millis/micros stuff and serial IO. There's no equivalent to routines such as the Arduino digitalRead or digitalWrite etc, I just do all that stuff via direct pin access, it's much faster and only marginally more difficult. And yes I should gather up the bits I have and put them in a repo somewhere, it's on my list I just haven't had the time to do it yet.

Re: Why I'm ditching the Arduino software platform

Interesting post. This, sadly, confirms much of my initial impressions of arduino libs.

(The IDE is a joke, it just isn't an IDE. Its handy for  using avrdude at the click of a button and the USB monitor window. That's about it).

I also thought it would be a good quick start method that I would replace with direct coding once I found my way around the platform. Big mistake. Learning to use the libraries just adds a second layer of stuff to learn. It more like doubles the task rather than making it "easy".

Probably the biggest and most inexcusable bug is the heap management that leaks like a broken sieve. This affects everything using strings and  probably relates to issues you mentioned in that area. Fortunately, Nick Gammon provides a good work around :

http://arduino.cc/forum/index.php/topic,95914.msg722258.html#msg722258

 

Arduino sketchland is good for blinking LEDs (like one or two) and that's about it.

Whenever I get a bug I'm not sure whether it's me being dumb or flakey arduino code and end up spending 10x more time checking the guts of their code as well as my own.

So, I'm coming to the same conclusion you did and ditching sketchland is becoming a necessity rather than a later option.

In the absence of a real IDE, serial IO 'print' lines are pretty essential.  Do you have anything of general use that you'd like to make available ?

 

LOL, my spelling checker wants to correct Arduino to arduous. I think it may be right.

 

Re: Why I'm ditching the Arduino software platform

Yes, I do have a from-the-ground-up serial library that sits underneath avr-libc stdio. It does interrupt-driven buffered input and output, and does basic input line buffering - backspace etc work, so you don't have to handle that in your code. And yes, I need to make it available somewhere ;-)

Re: Why I'm ditching the Arduino software platform

 I was really upset after delving more into the Arduino IDE and their libraries that it really can't do anything advanced. The attachInterrupt() method is total crap and works on polling, not interrupts. Serial data is TIME SENSITIVE and has to be dealt with immediately or information will get lost in transmission.

The only thing keeping me from switching entirely to something like eclipse (or just using np++ and the atmel compilers) is the Arduino analogWrite function. Writing a timer interrupt isr by hand is slightly more work than is needed for that functionality, and it doesn't seem to act like the String class or the Serial (class?) and include larger libraries when it's compiled. Plus it involves a bit of extra calculation, especially if you are making a specific PWM output. Any suggestions for replacing (or including) this functionality while tossing the Arduino library and IDE?

Re: Why I'm ditching the Arduino software platform

I don't have any specific libraries for doing the equivalent of analogWrite, generally I just set up the timers by hand. Once you've done it a couple of times it's not that hard, and you get better control. I've put all the bits and pieces I've written on sourceforge, at http://sourceforge.net/projects/abavr/

Re: Why I'm ditching the Arduino software platform

Use the PWM hardware, just like analogwrite does. No ISR required.

Re: Why I'm ditching the Arduino software platform

 Many of the above complaints are true. It is a hobbyist, amateur platform, not meant for propellor heads. But there are a number of good things happening. Codeblocks opensource IDE has been ported for Arduino, and is almost there. Also, even better, you can now get a free plugin for MSFT VisualStudio for arduino which is very neat, and reall compiles and uploads like a dream. Check out Visualmicro.com there is a real IDE complete with debugger.

Re: Why I'm ditching the Arduino software platform

 Also for free and with niiiice colours! MariaMole IDE for Arduino........

Re: Why I'm ditching the Arduino software platform

Your "String" test is way off-base; you've written a program using the Arduino libraries, but compiled it outside of the Arduino environment (the use of main() is a giveaway.)  The Arduino libraries make extensive use of the unused function garbage-collection in the linker (-ffunction-sections -fdata-sections -Wl,gc-sections)

Written as an Arduino sketch and compiled properly, the string example is less than 2k bytes (and that includes nearly 500 bytes of init code and such that your example may or may not have included.)

 

Re: Why I'm ditching the Arduino software platform

That's a misassumption I'm afraid, I used all those flags. Bear in mind this blog post is over a year old and was done using an older version of the Arduino environment, it's entirely possible things have improved.

Re: Why I'm ditching the Arduino software platform

Hi Alan,

I'm not sure how I even stumbled across your site, but glad I did.  I've only breezed through the last year of posts, but they're choc full of interesting links, not to mention the posts themselves.  I come from a 1990s microcontroller (Moto HC05 & HC11) programming-in-assembler background, & just recently 'reconnecting' with what's been going on in the last decade or so of my absence.  It's been fascinating to see how little has changed in the microcontroller world, & in the basic components used (other than 'new' things like internet connectivity, wireless options, & the ubiquity of C/C++/etc now, something I'll obviously have to come to terms with ;).

This post in particular piqued my interest.  I've been playing around with several people's rendition of the Philips TV 'Ambilight' concept on Arduino boards, and, besides finding a lot of half-baked code (both Arduino & Processing on the host to generate the RGB stream), even after "fixing it" I'm still experiencing "jittery" performance, and getting uncorrupted serial comms from Arduino back to the host serial-monitor has thus far been an exercise in whack-a-mole.  When I read that Arduino 1.0 was the first version to implement interrupt-driven serial send, I was... horrified.  What you've described here in the varuous Arduino libraries gives voice to my suspicion about relying on "someone else's stuff", especially when it's created in an open-source environment, something that can be either wonderful or infuriating & massively time-wasting - and sometimes both :).  That things like this are still so "wrong" despite Arduino's years of maturation is concerning, especially given the target market as you say, of neophytes.  You've given me a useful context on when it's appropriate to use Arduino code, and when it's better to roll one's own.  Thank you.

Re: Why I'm ditching the Arduino software platform

I've not regretted turning my back on the Arduino stuff, my requirements are generally pretty simple and now I've got replacements for the bits I did use the Arduino software doesn't offer me anything of benefit. The bits and bobs that I have written are available at http://sourceforge.net/projects/abavr/

Re: Why I'm ditching the Arduino software platform

 Hi Alan,

 

First of all thanks for your awesome blogs! They have been very helpful. I've been driving various led chipsets simply because I like toying with the Arduino and now I find myself pushing the limits of the Arduino Serial class. I feel comfortable with the Arduino library and as an experienced programmer I wouldn't mind some more insight into underlying mechanisms. I've played with some of your code on sourceforce with the ws2811. 

You have written the following:

<p style="margin-left: 40px;"><span style="font-family: 'Times New Roman'; font-size: medium; background-color: rgb(240, 240, 240);">What I have isn't a set of patches to the Arduino environment, it's a small library of routines that I use as a replacement for the bits of the Arduino libraries that I was still actually using - mostly the millis/micros stuff and serial IO. There's no equivalent to routines such as the Arduino digitalRead or digitalWrite etc, I just do all that stuff via direct pin access, it's much faster and only marginally more difficult. And yes I should gather up the bits I have and put them in a repo somewhere, it's on my list I just haven't had the time to do it yet.</span>

Now, would you be interested in sharing that code with the community? Or parts of it? I am having performance issues and glitches when I update my data thoughput. I am sure your implementation will be of great benefit to those in a similar situation.

Re: Why I'm ditching the Arduino software platform

Great to hear from you and good to hear this is on your list. I don't get to play with my Arduino every day so I will try to visit your blog every so often. Thanks again ..

Re: Why I'm ditching the Arduino software platform

Hi Eric,

You might want to check out 'FastPin', http://waitingforbigo.com/2013/02/19/introducing-fastpin-the-low-level-heart-of-fastspi_led2/

Re: Why I'm ditching the Arduino software platform

I agree with all your points and have come to the same conclusion. To fix some of the issues and show how to write industry strength code that scales, is faster, smaller foot-print, I have written Cosa. It is now over 50 KLOC with over 100 classes and 130 plus example sketches.

https://github.com/mikaelpatel/Cosa

The Arduino boards are highly available even though need to be modified for low power applications. The IDE is too slow for me but ok for the build but the Arduino/Wiring core library is poorly written code mostly by students.

Cheers!

Re: Why I'm ditching the Arduino software platform

Hi Mikael,

Small world.  I was just reading the SPI thread on the Arduino dev forum, & Paul S complimenting your Cosa code.  I look forward to checking it out.

 

Re: Why I'm ditching the Arduino software platform

I'm not surprised!

For real deterministic real-time stuff, most professional either developed their custom routines, or used "validated" library.  But the bloated code for most hobbyist is probably adequate.

 

Re: Why I'm ditching the Arduino software platform

Just came across your page while researching the Serial Interrupt.

Believe it or not the serialEvent() function is not called in response to a character arriving on the Serial input

The docs on the Arduion site   http://arduino.cc/en/Reference/SerialEvent   are completely wrong.

This function is actually called by the Arduino code that is executed at the of each loop() function.

i.e in the main() Arduino function it basically does

setup();

while(1)

{

loop();

serialRun()  ;// This calls serialEvent if any data in the serial buffer

};

 

Hence serialEvent is completely useless.

 

So I'm now researching how to write my own serial interrupt handler.

 

 

Re: Why I'm ditching the Arduino software platform

Hi Roger, I've already implemented something similar, to provide an interrupt-driven serial driver for avr-libc's stdio implementation. See http://sourceforge.net/p/abavr/lib/ci/default/tree/utils/serial.c, if it isn't useable directly it should at least give you a worked example to use - it isn't that hard to get interrupt-driven serial IO working.