Tuesday, March 12, 2013

In a comment in programmers.SE, Dibbeke states the following:

Small hint here: your most important liability in programming is your code. So, less is truly more here.

Many people view code as an asset. It is not. Code is a liability. The functionality code provides is the asset. So, it follows that you should try to get that functionality with the minimum amount of code. Sometimes, you should even forgo some functionality, because its cost in code (and the cost of code is _not_ the cost of writing it) exceeds the value it provides.

See also: SoftwareAsLiability and LegacyCode at the c2.com wiki.

Thursday, November 17, 2011

Your own linker warnings using the GNU toolchain

You've probably seen linker warnings like these:

linkwarnmain.c:(.text+0x1d): warning: foo is deprecated, please use the shiny new foobar function
linkwarnmain.c:(.text+0x27): warning: the use of `tmpnam' is dangerous, better use `mkstemp'

These warnings are actually stored in ELF sections named .gnu.warning.symbolname:

$ objdump -s -j .gnu.warning.gets /lib/libc.so.6

/lib/libc.so.6:     file format elf32-i386

Contents of section .gnu.warning.gets:
 0000 74686520 60676574 73272066 756e6374  the `gets' funct
 0010 696f6e20 69732064 616e6765 726f7573  ion is dangerous
 0020 20616e64 2073686f 756c6420 6e6f7420   and should not
 0030 62652075 7365642e 00                 be used..

So when you try to use a symbol, if the linker sees a section whose name matches the above pattern, it emits the corresponding warning message.

You can add your own linker warnings to your source files:

void foo(void)

static const char foo_warning[] __attribute__((section(".gnu.warning.foo"))) =
        "foo is deprecated, please use the shiny new foobar function";

void foo(void);

int main()

        return 0;

Then, when you compile, you will get a warning:

$ gcc -Wall -c linkwarn.c
$ gcc -Wall -o linkwarnmain linkwarnmain.c linkwarn.o
/tmp/ccyHLTw6.o: In function `main':
linkwarnmain.c:(.text+0x1d): warning: foo is deprecated, please use the shiny new foobar function

glibc machinery for emitting linker warnings is a little bit more complicated:

#ifdef HAVE_ELF

/* We want the .gnu.warning.SYMBOL section to be unallocated.  */
#  define __make_section_unallocated(section_string)    \
  asm (".section " section_string "\n\t.previous");
#  define __make_section_unallocated(section_string)    \
  asm (".pushsection " section_string "\n\t.popsection");
# else
#  define __make_section_unallocated(section_string)
# endif

/* Tacking on "\n\t#" to the section name makes gcc put it's bogus
   section attributes on what looks like a comment to the assembler.  */
#  define __sec_comment "\"\n\t#\""
# else
#  define __sec_comment "\n\t#"
# endif
# define link_warning(symbol, msg) \
  __make_section_unallocated (".gnu.warning." #symbol) \
  static const char __evoke_link_warning_##symbol[]     \
    __attribute__ ((used, section (".gnu.warning." #symbol __sec_comment))) \
    = msg;

The warning message is marked as used so the optimizer doesn't decide to completely optimize it out of existence. Also, the section is not marked as allocatable to prevent the loader from loading it into memory. Since gcc's section attribute doesn't allow to change its default flags, the solution is to declare the section using asm, return to the previous section (the one the compiler was using beforehand), and prevent (via __sec_comment) the assembler from seeing the flags that gcc adds in its section attribute output. Otherwise we'd have something similar to:

$ gcc -Wall -S linkwarn.c
$ cat linkwarn.s
        .file   "linkwarn.c"
.globl foo
        .type   foo, @function
        pushl   %ebp
        movl    %esp, %ebp
        popl    %ebp
        .size   foo, .-foo
        .section        .gnu.warning.foo,"a",@progbits
        .align 32
        .type   foo_warning, @object
        .size   foo_warning, 60
        .string "foo is deprecated, please use the shiny new foobar function"
        .section        .note.GNU-stack,"",@progbits
        .ident  "GCC: (GNU) 3.4.5 (Gentoo 3.4.5-r1, ssp-3.4.5-1.0, pie-8.7.9)"

... where the section is marked as allocatable via the "a" flag.

Tuesday, October 18, 2011

Terminals in *nix

Terminals were hardware devices that consisted of a keyboard and an output device (initially a line printer, later a CRT monitor). A large computer could have several remote terminals connected to it. Each terminal would have a protocol for communicating efficiently with the computer, for CRT-based terminals this includes having special "control sequences" to change cursor position, erase parts of the current line/screen, switch to an alternate full-screen mode, ...

A terminal emulator is an application emulating one of those older terminals. It allows to do functions like cursor positioning, setting foreground and background colors, ... Terminal emulators try to emulate some specific terminal protocol, but each has its own set of quirks and deviations.

Unix systems have databases describing terminals and terminal emulators, so applications are abstracted away from the particular terminal (or terminal emulator) in use. An older database is termcap(5), while terminfo(5) is a newer database. These databases allow applications to query for the capabilities of the terminal in use. Capabilities can be booleans, numeric capabilities, or even string capabilities, e.g.: if a specific terminal type has/supports a F12 key, it will have a capability "key_f12" (long terminfo name), "kf12" (short terminfo name), "F2" (termcap name) describing the string that key produces. Try it with: tput kf12 | od -tx1.

Since programming directly with capabilities can be cumbersome, applications typically use a higher-level library like curses/ncurses, slang, etc...

There is a special environment variable called TERM that tells applications what terminal type they are talking to. This variable should be set to the exact terminal type if it exists in the database, for best results. This just tells the application which precise protocol and protocol deviations does the terminal understand. Changing the TERM variable does not change the terminal type, it just changes the terminal type the application thinks it is talking to.

Monday, October 17, 2011


I remember SuSE Linux had a motd that simply said: "Have fun..." That has always been the spirit of Unix, having an environment in which it is fun to work and tinker about. We owe that to Ken Thompson, Dennis Ritchie and their colleagues at Bell Labs (http://www.princeton.edu/~hos/Mahoney/unixpeople.htm).

Dennis Ritchie also evolved Thompson's B into C, giving us a concise and practical systems programming language. Kernighan and Ritchie taught many people how to write great code, some of those people taught other people, directly, or simply by sharing great code. The end result has been a ripple effect that has had a profound and positive impact on how many of us code. Not many people in history have influenced the work of others so much.

Dennis Ritchie is no longer with us, he will be greatly missed. He is among those luminaries we will never forget. The computing community will never forget Jon Postel, Richard Stevens and Dennis Ritchie.


Monday, October 10, 2011

Tentative Definitions in C

Suppose you want to access a variable from several translation units. You would declare it in a header file:

int foo;

You would optionally define it on a translation unit:

int foo = 2;

And you would refer to it on some other(s) translation unit(s):

#include <stdio.h>

#include "example.h"

int main()
        printf("foo = %d\n", foo);

        return 0;

If we compile and run the above we get:

$ gcc -Wall -o example example-main.c example.c
$ ./example
foo = 2

int foo; from header.h is a tentative definition. If you comment out the definition in example.c, the tentative definition acts as a definition with an initializer of 0:

$ gcc -Wall -o example example-main.c example.c
$ ./example
foo = 0

But it can be considered to be better style to restrict header files to contain declarations, and not definitions, not even tentative ones. In that case, we would add a extern storage specifier, which would turn our declaration on the header file from a tentative definition to a mere declaration:

extern int foo;

In this case, if we don't define foo in any translation unit, the linker will error out:

$ gcc -Wall -o example example-main.c example.c
/tmp/ccS3FMxx.o: In function `main':
example-main.c:(.text+0x1d): undefined reference to `foo'
collect2: ld returned 1 exit status

All of the above is for C. C++ doesn't have tentative declarations, and therefore has some different rules on what constitutes a definition. In C++, int foo; constitutes a definition, while extern int foo; constitutes a declaration. So, if we leave out the extern storage specifier in C++, the linker will error out:

$ g++ -Wall -o example example-main.cpp example.cpp
/tmp/cc8fzx4x.o:(.data+0x0): multiple definition of `foo'
/tmp/ccxLfXPo.o:(.bss+0x0): first defined here
collect2: ld returned 1 exit status

Friday, September 23, 2011

Spring @Transactional Propagation Reference

Spring's propagation types for the @Transactional annotation differ in how they behave in presence/absence of an existing transaction:

Propagation types and their behaviour
PROPAGATION TYPE no current transaction there's a current transaction
MANDATORY throw exception use current transaction
NEVER don't create a transaction, run method outside any transaction throw exception
NOT_SUPPORTED don't create a transaction, run method outside any transaction suspend current transaction, run method outside any transaction
SUPPORTS don't create a transaction, run method outside any transaction use current transaction
REQUIRED (default) create a new transaction use current transaction
REQUIRES_NEW create a new transaction suspend current transaction, create a new independent transaction
NESTED create a new transaction create a new nested transaction

It could be argued that NEVER and NOT_SUPPORTED (and arguably SUPPORTS also) don't serve any useful purpose, and exist only for completeness; after all, why would you want to explicitly run something outside of any transaction?

As for the other propagation types, REQUIRED/MANDATORY/SUPPORTS would be used when you want your code to be part of the current transaction, sharing its commit/rollback fate; NESTED would come into use in cases where you want to be able to rollback only parts of a bigger transaction; and finally REQUIRES_NEW is to be used when you want to do something totally unrelated to the current transaction.

The difference between REQUIRED, MANDATORY and SUPPORTS strives in how they behave when there is no existing transaction. MANDATORY acts as a sort of assert directive, ensuring there is an opened transaction. REQUIRED creates a new transaction, being safe to use outside any transaction. SUPPORTS rides along an existing transaction, but doesn't create a new one if none exists.

Monday, July 11, 2011

Introduction to regexes

Let Σ be an alphabet.

  • ε (the empty string) is a regular expression.
  • If a is a symbol from the alphabet Σ, a is a regular expression.
  • If a and b are symbols from the alphabet Σ, ab (a and b concatenated) is a regular expression.
  • If a and b are symbols from the alphabet Σ, a|b (a or b) is a regular expression.
  • If a is a symbol from the alphabet Σ, a* (a repeated 0 or more times) is a regular expression.
A regular language is the language defined by a regular expression. A regular language is the language matched by a finite automaton. A word w from a regular language can be recognized in O(n).

Unix regexes:

  • . means any character.
  • Concatenation is represented without any symbol.
  • Union (alternation) is represented by |.
  • * means Kleene closure (0 or more repetitions).
  • + means 1 or more repetitions.
  • ? means 0 or 1 repetition.
  • ^ means start of line.
  • $ means end of line.
  • \< means start of word (in some programs).
  • \> means end of word (in some programs).
  • {n} means n repetitions (in some programs).
  • {,m} means from 0 to m repetitions (in some programs).
  • {n,m} means from n to m repetitions (in some programs).
  • a|b|c|d|e|f can be written as [abcdef] or [a-f].
  • a|b|c|d|h|i can be written as [a-dhi] or [a-dh-i], ...
  • [^list] matches characters not in list.
  • () is used for grouping.
  • \ escapes special meaning of a metacharacter, e.g.: \*
Precedence is: repetition, concatenation, union.

The supported regular expression syntax varies among programs, (e.g: gawk vs mawk, vi vs vim, ...).


Many programs allow the use of \n (where n is a number) to match a subexpression enclosed in () or \(\), e.g: in vim, \<\(.\)\(.\).\2\1\> matches five-letter palindromes. Regular expressions using these back-references are no longer true regular expressions, and not all of them can be matched in O(n) time.

Regex/Regexp vs regular expressions:

Since modern regexes match things that cannot be matched with true regular expressions, it's been proposed (by Larry Wall) to call them regexes, not regular expressions. Perl regexes have been gaining expressive power through the years, and have grown, among many other things, support for look-around assertions, support for calling Perl code, and support for recursion. Many regex engines have taken things from Perl's regexes. On the other hand, Google's re2 library uses true regular expressions and a DFA-based implementation (in contrast to the NFA-based implementation of backtracking-regex engines) to achieve very high performance.