Category Archives: programming
Somebody Else’s Problem
The late, great Douglas Adams described the “SEP field”. Something may be huge and blindingly obvious and right in front of you, but it’s somebody else’s problem so you don’t see it.
I’ve seen a bug report for a cryptographic function. Nothing too unusual there you might think, but there’s a twist. Any fix we could ever implement would not be a fix, but rather an ugly hacked workaround for somebody else’s problem.
Let me explain. The bug concerns memory that may have contained secrets. To free it after use, we wipe the potentially-sensitive data:
memset(buf, 0, bufsize);
free(buf);
Now the bug there isn’t in our code, it’s in the fact that some compilers and settings might optimise out the memset leaving the sensitive data in freed memory.
How to fix that?
Introduce dummy code that re-uses the buffer after memset? Not just horribly inefficient, but difficult to guarantee that can’t also be optimised out, either by the compiler or by another developer who sees this useless code.
OK, what about a nightmare of #ifdefs to target compilers? Erm, no thanks. If something’s in a widely-supported standard like ANSI or POSIX, the last thing we want is to replace it with a maintenance nightmare of proprietary hacks.
So fix the build to disable optimisation compiling the source file in question? Not really a solution: downstream folks building and distributing the software may have their own build setups independent of ours.
An altogether better solution-in-principle would be some #pragma that could be standardised across compilers and could disable optimisations within a source file. It could maybe affect the entire source file that declares it, or ideally it could scope arbitrary sections of code in the manner of #if / #ifdef. But that’s Somebody Else’s Problem.
tengine
How well can open source transcend cultural and language barriers?
A few days ago I posted to the ironbee-devel list about the experimental nginx module for Ironbee. It cannot be incorporated into Ironbee’s normal build/test processes in the manner of the Apache HTTPD and Traffic Server modules, because nginx doesn’t support loadable modules, so the libraries have to be installed so they can be linked when the nginx module is built. This, along with a much more limited API, is presumably one of the design decisions the nginx team made when they focussed firmly on performance over extensibility.
In response to my post, someone drew my attention to an nginx fork called tengine. The key point of tengine is that it addresses precisely the issue of loadable modules. And not just that: it supports input filters, opening up the possibility of overcoming another shortcoming of the nginx module – the need to read and buffer an entire request body before scanning it. Interesting.
I’ve now downloaded tengine, and tried building the nginx-ironbee module for it. It appears to be fully API-compatible, and the only source change needed arose from their only having forked nginx 1.2.6 (the stable version), whereas I had developed the ironbee module using nginx 1.3.x. All I need to add is a preprocessor directive to detect nginx version and work around the missing API, and the two are (or appear to be) fully interchangeable (well, until I take advantage of input filtering to improve it further). This is seriously useful!
Tengine has been a collaborative open source effort for two years now (that’s almost as long as TrafficServer), yet this is the first I’d heard of it! Perhaps one reason for that is that Tengine is made-in-China. Just as TrafficServer originated from a single major site (Yahoo) before being open-sourced, so Tengine originates with Taobao and a Chinese developer community. They have English-language resources including a decent-enough website. But as a developer I want the mailinglist: there is an English-language list, but just looking at archive sizes tells me all the traffic takes place on the corresponding chinese-language list.
How much of a barrier is language? I’ve written about that before, and now it’s my turn to find myself the wrong side of a language barrier. Actually that applies to nginx too: the Russian-born web server has a core community whose language I don’t speak. Developing the nginx-ironbee module gave me an opportunity to test a barrier from the outside, and I’m happy to report I got some helpful responses and productive technical discussion on nginx’s English-language developer list. A welcoming community and no language barrier to what I was doing.
Like other major open source projects, nginx has achieved a critical mass of interest that makes it not merely possible but inevitable that it crosses language barriers. Not all nginx’s Russian core team participate in English-language lists (nor should they!), but all it takes is one or two insiders with fluent English as points of contact to bridge the divide. I’ve no idea if I’ll get a good experience on tengine’s english-language list, but I expect I’ll find out now that I’ve heard of tengine and find it meets a need.
Corollary: there is still a language barrier. Of course! With Apache I started out developing applications (some of them modules) before making the transition to the core developer team. With nginx or tengine I know I can’t make that transition – at least not fully. And because I know that, I’m unlikely to let my work take me in that direction. The same kind of consideration may or may not have led the tengine team to fork rather try and work directly with nginx.
Luddites or weenies?
Tim Bray: “here it is nearly thirty years into my programming career and I’m still debugging with print statements“. That didn’t jump out at me, but Ian Murdock saw it, and agrees. In such distinguished company as Tim and Ian, I think I can admit to my own luddite tendencies with a shell, vi, and a whole bunch of commandline utilities like make and svn. At the same time, I take much more advantage of a decent GUI than most, which is why I hate restrictive, labour-intensive desktop environments like Windows[1] and Mac[1].
As for debugging, yes, I sometimes use print statements. Where I can get away with it I prefer something more interactive like gdb, and even a level up from there a thin GUI is nice (whatever happened to xdbx? A google search finds no new version since 1992, a very long time before gdb could be said to be a serious competitor to dbx).
From time to time I do use a full-blown IDE. On Windows I use MS Visual studio, not out of choice but because there’s no half-decent commandline environment. While it has some useful features, I find these heavily outweighed by labour-intensive hoops I have to jump through to import a project into the studio, the lack of control, etc. I use cygwin (the primitive best-available shell) alongside it, and heartily curse the defective GUI that won’t let me have both visible and usable at the same time on any realistic size of desktop.
When I first programmed Java in the 1990s I used Sun’s studio and workshop projects, and they certainly helped bootstrap my knowledge of the language. But within a couple of months I’d reverted to the commandline toolkit for getting any real work done. The nice GUIs are good for playing with prototypes and (in cases like glade) creating some really useful skeleton code. But when GUI becomes IDE, it tends to get way too restrictive for real work.
Which leads me to wonder: where’s the middle-ground in development? There’s the choice of IDE or DIY toolkit, and of course there’s the DIY IDE (aka emacs). But what happened to the idea of a tool that simplifies things that can be simplified, without putting the programmer in a straitjacket? What happened to simple but useful productivity tools like xdbx, or is gdb (wrapped in a GUI or otherwise) really the only show in town?
Ho, hum. This feels like a “getting old” rant. On reflection, this middle ground is there, and I’ve even named some of the tools, and what I’m missing is an IDE I can live with on a regular basis. Wonder how I’d feel if I were thrown back to a 1980s toolkit and VT100 terminal? Or even the dyslexic teletype and line editor I had to work on in my first job after graduating?
[1] Just to list a couple of defects shared by both those platforms:
- No way to use a window without raising it. If your reaction to that is “so what?”, see Steve Yegge’s explanation of what you’re missing.
- No mouse-only cut&paste. Keyboard sequences like ctrl-c/ctrl-v mean you either tie up both hands in a simple operation, or abandon the mouse altogether. I can respect the latter (use it myself, in forms like vi’s y/p[2], though not across different applications and windows), but the former is just dreary, distracting, time-consuming, error-prone faff.
[2] vi’s keystrokes are also vastly less effort for clumsy hands than anything requiring multi-key combos like [alt|meta|ctrl]+action.
niq's soapbox 