Netmap, a high-performance packet send/receive framework for FreeBSD

Luigi Rizzo has recently released Netmap, a high performance packet processing framework for FreeBSD. Its generic approach make it easily portable to various NIC drivers and even other operating systems. A libpcap wrapper and a Linux port are planned. The complete paper describing netmap is available here.

RSS (Receive-Side Scaling) with Myricom Sniffer 10G

Recently, while implementing PacketDam for a customer company on top of the excellent Myri-10G card, I encountered a minor nuisance: the default hashing algorithm that splits the incoming traffic among cores uses both the source and destination IP addresses. Therefore, packets heading towards the same destination from different source addresses end up in different queues. That isn’t a huge problem for most packet analysis software, but every PacketDam instance needs to see all traffic belonging to a given destination in order to work properly.

Luckily, Myricom has been kind enough to provide a packet sniffing API (which is also neatly wrapped in recent versions of libpcap) which enables the programmer to fine tune the packet hashing. To use this feature with any pcap-based application, one needs to patch libpcap’s pcap-snf.c as follows:

• Declare a custom struct snf_rss_params:

• Optionally overwrite the environment:

• Pass the custom hashing function pointer to snf_open() by filling snf_rss_params appropriately:

• Implement your custom hashing function:

The SNF library then applies a modulo (number of cores) to the return value to obtain the queue index for the current packet. The above function will yield an uneven packet distribution. A non-cryptographic hashing function (I prefer MurmurHash, and so does the Myricom firmware) can be used to spread the load equally across the cores, as well as for mapping IPv6 destination addresses.

Mitigating web attacks with Varnish

I’ve been a Varnish user and enthusiast for quite some time, ever since the 1.x days. Perhaps also because of my FreeBSD bias.

Performance-wise, Varnish has always been a treat, successfully replacing expensive solutions from big vendors like BlueCoat (who also run a modified FreeBSD) or Crescendo Networks. Tie it to OpenBSD’s packet filter and the nginx web server and you get an excellent HTTP stack.

A couple of days ago, while I was mangling HTTP headers inside Varnish in order to prevent web attacks, it occured to me that somebody might have put together something more consistent than my quick’n dirty setup. And indeed, it’s all there, in Kacper Wysocki’s GitHub repository.

Recording good guitar sounds at home, part 2

A kind gentleman on a guitar forum I read has linked me to this article by Craig Anderton on how to get better sounds out of software amp simulators. He explains how to dial out unwanted frequencies in order to make the simulator sound closer to what you’re used to expect from an actual amp.  Make sure you listen to the provided sound samples, they illustrate each step of the process.

This method can be used instead of the Impulse-Response cabinet emulation technique which I described in my first post, or even along with it, depending on one’s taste.

Thanks, Craig!

PacketDam is launched

It’s been quite a while since I wrote PacketDam, so I decided it deserved its own website and some advertising. There are a few Romanian companies using it and they all got to know about it through word of mouth. I guess it’s just one of those products that sell themselves

Without further ado, here is PacketDam, a cost-effective software solution against DDoS.

kqueue(2) support for JDK/OpenJDK on FreeBSD

David Xu of FreeBSD fame has recently added support for kqueue(2) to Java. Both Sun and OpenJDK ports have been patched. Although not yet enabled by default, this should definitely give a boost to applications relying on NIO. It would be interesting to see how Openfire scales with this. However, they already have a NIO-like API implemented on top of MINA. Right now I wish it hadn’t been so long since I last touched Java.

Apple’s Java distribution has been supporting it for quite a while, but apparently their kqueue(2) is broken, according to several reports I’ve found on various mailing lists, particularly when watching descriptors associated to files. I’ve only used it with sockets, so I can’t complain.

kqueue is by far my favorite I/O multiplexing API. Besides storing user data in the kevent structure (which I usually use for storing callbacks or object pointers), it also submits and retrieves multiple events in a single system call. I’ve always wondered why epoll(4) in Linux didn’t use a similar approach.