Bluecherry is hiring an Ubuntu developer!

My old employer, Bluecherry, is looking to hire an Ubuntu developer with the following experience:

• Extensive knowledge of the Video4Linux2 and ALSA sound API
  
• Prior experience in Linux based software design / implementation
  
• Prior knowledge of Ubuntu, including building / maintaining Debian packages
  
• Prior experience with gstreamer and RTSP
  
• Prior experience with MYSQL.
  
• Prior experience with Matroska file containers and video encoding
  
• Excellent verbal and written communication skills
  
• Strong knowledge of C
  
• Previous work with and understanding of working with video / audio formatting / codecs including MPEG4 and H.264
  
• Internet and operating system security fundamentals
  
• Sharp analytical abilities and proven design skills
  
• Strong sense of ownership, urgency, and drive
  
• Demonstrated ability to achieve goals in a highly innovative and fast paced environment
  

Yes it's a long list, but you are replacing me, so get used to high expectations, but a highly rewarding job. You don't need to know all of this, but you should have a good enough foundation that you can embrace the current codebase and not shy away from a steep learning curve. Visit Bluecherry's job posting for more information.

Using your new Bluecherry MPEG-4 codec card and driver...

Now that the dust has settled and people are taking notice of the new driver for Bluecherry's MPEG-4 codec cards, here's a quick How-To for using it.

You will notice that there are two types of v4l2 devices created for each card. One device for the display port that produces uncompressed YUV and one for each input that produces compressed video in either MPEG-4 or MJPEG.

We'll start with the display port device. When loading the driver, a display port is created as the first device for that card. You can see in dmesg output something like this:

solo6010 0000:03:01.0: PCI INT A -> GSI 22 (level, low) -> IRQ 22
solo6010 0000:03:01.0: Enabled 2 i2c adapters
solo6010 0000:03:01.0: Initialized 4 tw28xx chips: tw2864[4]
solo6010 0000:03:01.0: Display as /dev/video0 with 16 inputs (5 extended)
solo6010 0000:03:01.0: Encoders as /dev/video1-16
solo6010 0000:03:01.0: Alsa sound card as Softlogic0

This is for a 16-port card. The output for a 4-port card would show "Encoders as /dev/video1-4" and similarly for 8-port show /dev/video1-8.

The display port allows you to view and configure what is shown on the video out port of the card. The device has several inputs and depends on which card you have installed:

• 4-port: 1 input per port and 1 virtual input for all 4 inputs in 4-up mode.
• 8-port: 1 input per port and 2 virtual inputs for 4-up on inputs 1-4 and 5-8 respectively.
• 16-port: 1 input per port and 5 virtual inputs for 4-up on inputs 1-5, 5-8, 9-12 and 13-16 and 1 virtual input for 16-up on all inputs.

You do not have to open this device for the video output of the card to work. If you open the device and set the input to input 2, and close it (without viewing any of the video) it will continue to show that input on the video out of the card. So you can simply change inputs using v4l2's ioctl's.

This is useful if you want to have live display to a CRT and use a simple program that rotates through the inputs (or multi-up virtual inputs) a few second intervals.

You can still use vlc, mplayer or whatever to view this device (you can open it multiple times).

Now for the encoder devices. There's obviously one device for each physical input on the card. The driver will allow you to record MPEG-4 and MJPEG from the same device (but you must open it twice, one for each feed). The video format cannot be configured once recording starts. So if you open the device for MPEG-4 and full D1 resultion at 30fps, that's what you're going to get if you also open a simultaneous record for MJPEG.

However, it's good to note here that MJPEG will automatically skip frames when recording. This allows you to pipe the output to a network connection (e.g. MJPEG over HTTP) with no worry of the remote connection being overloaded on bandwidth.

However, this isn't so for MPEG-4. It is possible if you are too slow at recording (not likely) to fall behind the card's internal buffer. I was not able to do this writing the full frames to disk on 44 records (4 cards of 16, 16, 8 and 4 ports).

Unlike any card before this supported by v4l2, the Bluecherry cards produce containerless MPEG-4 frames. Most v4l2 applications expect some sort of MPEG-2 stream such as program or transport. Since these programs do not expect MPEG-4 raw frames, I don't know of any that are capable of playing the encoders directly (much less being able to record from them). You can do something simple like 'cat /dev/video1' and somehow pipe it to vlc (I haven't tested this), or write a program that just writes the frames to disk (I have tested this, most programs can play the raw m4v files produced from the driver).

However, since most people will record to disk, the easiest way is to write the video frames straight out to disk.

Now on to the audio. The cards produce what is known as G.723, which is a voice codec typically found on phone systems (especially VoIP).

Since Alsa currently doesn't have a format for G.723, the driver shows it as unsigned 8-bit PCM audio. However, I can assure you that it isn't. I have sent a patch that was included in alsa-kernel (hopefully getting synced to mainline soon). But this only defines the correct format, it doesn't change the way you handle it at all.

You must convert G.723-24 (3-bit samples at 8khz) yourself. The example program I provide in my next post will show you how to do this, as well as how to convert it to MP2 audio, and record all of this to a container format on disk for later playback.

Softlogic 6010 4/8/16 Channel MPEG-4 Codec Card Driver Released

As I've talked about before, the company I work for has been dedicated to producing stable video surveillance products based on Linux.

Bluecherry's primary device for their video surveillance applications is the Softlogic based MPEG-4 codec card, which is available in 4, 8 and 16 channel models. The original driver for this card, although available as Open Source, was pretty pathetic to say the least. Most of it was just a kludge of the Windows driver, exposing all of the functionality, but with little effort to make it Linux savvy.

That's where I came in. I've since rewritten the driver so that it makes use of Linux's Video4Linux2 and Alsa driver API's. It's currently 90% functional, and many times more efficient than the original OEM driver.

Here is a quick run-down of some of the features and plus-ones against the original driver:

• Video4Linux2 interface allows easy use of existing capture software
• Alsa interface allows for easy audio capture (however, see G.723 caveats from my previous posts)
• Zero-copy in the driver. The original driver DMA'd and then copied the MPEG frames to userspace. The new driver makes use of v4l2 buffers and can DMA directly to an MMAP buffer for userspace.
• Simultaneous MPEG/MJPEG feed per channel, selectable via v4l2 format
• Standard v4l2 uncompressed video YUV display with multi-channel display format (4-up)

Now that the driver is nearing completion, it's about time to release it. I've done so via Launchpad.

• PPA Archive
• Tarball
• git clone git://git.kernel.org/pub/scm/linux/kernel/git/bcollins/solo6x10

If you are on an Ubuntu system, you can install the DKMS package from the PPA archive using these commands:

sudo add-apt-repository ppa:ben-collins/solo6x10
sudo apt-get update
sudo apt-get install solo6010-dkms

Note, I've only supplied this for Lucid right now, but if you download the .deb or the .tar.gz you should be able to install it on any recent kernel.

Writing an ALSA driver: PCM handler callbacks

So here we are on the final chapter of the ALSA driver series. We will finally fill in the meat of the driver with some simple handler callbacks for the PCM capture device we've been developing. In the previous post, Writing an ALSA driver: Setting up capture, we defined my_pcm_ops, which was used when calling snd_pcm_set_ops() for our PCM device. Here is that structure again:

static struct snd_pcm_ops my_pcm_ops = {
        .open      = my_pcm_open,
        .close     = my_pcm_close,
        .ioctl     = snd_pcm_lib_ioctl,
        .hw_params = my_hw_params,
        .hw_free   = my_hw_free,
        .prepare   = my_pcm_prepare,
        .trigger   = my_pcm_trigger,
        .pointer   = my_pcm_pointer,
        .copy      = my_pcm_copy,
};

First let's start off with the open and close methods defined in this structure. This is where your driver gets notified that someone has opened the capture device (file open) and subsequently closed it.

static int my_pcm_open(struct snd_pcm_substream *ss)
{
        ss->runtime->hw = my_pcm_hw;
        ss->private_data = my_dev;

        return 0;
}

static int my_pcm_close(struct snd_pcm_substream *ss)
{
        ss->private_data = NULL;

        return 0;
}

This is the minimum you would do for these two functions. If needed, you would allocate private data for this stream and free it on close.

For the ioctl handler, unless you need something special, you can just use the standard snd_pcm_lib_ioctl callback.

The next three callbacks handle hardware setup.

static int my_hw_params(struct snd_pcm_substream *ss,
                        struct snd_pcm_hw_params *hw_params)
{
        return snd_pcm_lib_malloc_pages(ss,
                         params_buffer_bytes(hw_params));
}

static int my_hw_free(struct snd_pcm_substream *ss)
{
        return snd_pcm_lib_free_pages(ss);
}

static int my_pcm_prepare(struct snd_pcm_substream *ss)
{
        return 0;
}

Since we've been using standard memory allocation routines from ALSA, these functions stay fairly simple. If you have some special exceptions between different versions of the hardware supported by your driver, you can make changes to the ss->hw structure here (e.g. if one version of your card supports 96khz, but the rest only support 48khz max).

The PCM prepare callback should handle anything your driver needs to do before alsa-lib can ask it to start sending buffers. My driver doesn't do anything special here, so I have an empty callback.

This next handler tells your driver when ALSA is going to start and stop capturing buffers from your device. Most likely you will enable and disable interrupts here.

static int my_pcm_trigger(struct snd_pcm_substream *ss,
                          int cmd)
{
        struct my_device *my_dev = snd_pcm_substream_chip(ss);
        int ret = 0;

        switch (cmd) {
        case SNDRV_PCM_TRIGGER_START:
                // Start the hardware capture
                break;
        case SNDRV_PCM_TRIGGER_STOP:
                // Stop the hardware capture
                break;
        default:
                ret = -EINVAL;
        }

        return ret;
}

Let's move on to the handlers that are the work horse in my driver. Since the hardware that I'm writing my driver for cannot directly DMA into memory that ALSA has supplied for us to communicate with userspace, I need to make use of the copy handler to perform this operation.

static snd_pcm_uframes_t my_pcm_pointer(struct snd_pcm_substream *ss)
{
        struct my_device *my_dev = snd_pcm_substream_chip(ss);

        return my_dev->hw_idx;
}

static int my_pcm_copy(struct snd_pcm_substream *ss,
                       int channel, snd_pcm_uframes_t pos,
                       void __user *dst,
                       snd_pcm_uframes_t count)
{
        struct my_device *my_dev = snd_pcm_substream_chip(ss);

        return copy_to_user(dst, my_dev->buffer + pos, count);
}

So here we've defined a pointer function which gets called by userspace to find our where the hardware is in writing to the buffer.

Next, we have the actual copy function. You should note that count and pos are in sample sizes, not bytes. The buffer I've shown we assume to have been filled during interrupt.

Speaking of interrupt, that is where you should also signal to ALSA that you have more data to consume. In my ISR (interrupt service routine), I have this:

snd_pcm_period_elapsed(my_dev->ss);

And I think we're done. Hopefully now you have at least the stubs in place for a working driver, and will be able to fill in the details for your hardware. One day I may come back and write another post on how to add mixer controls (e.g. volume).

Hope this series has helped you out!

<< Prev

Writing an ALSA driver: PCM Hardware Description

Welcome to the fourth installment in my "Writing an ALSA Driver" series. In this post, we'll dig into the snd_pcm_hardware structure that will be used in the next post which will describe the PCM handler callbacks.

Here is a look at the snd_pcm_hardware structure I have for my driver. It's fairly simplistic:

static struct snd_pcm_hardware my_pcm_hw = {
        .info = (SNDRV_PCM_INFO_MMAP |
                 SNDRV_PCM_INFO_INTERLEAVED |
                 SNDRV_PCM_INFO_BLOCK_TRANSFER |
                 SNDRV_PCM_INFO_MMAP_VALID),
        .formats          = SNDRV_PCM_FMTBIT_U8,
        .rates            = SNDRV_PCM_RATE_8000,
        .rate_min         = 8000,
        .rate_max         = 8000,
        .channels_min     = 1,
        .channels_max     = 1,
        .buffer_bytes_max = (32 * 48),
        .period_bytes_min = 48,
        .period_bytes_max = 48,
        .periods_min      = 1,
        .periods_max      = 32,
};

This structure describes how my hardware lays out the PCM data for capturing. As I described before, it writes out 48 bytes at a time for each stream, into 32 pages. A period basically describes an interrupt. It sums up the "chunk" size that the hardware supplies data in.

This hardware only supplies mono data (1 channel) and only 8000HZ sample rate. Most hardware seems to work in the range of 8000 to 48000, and there is a define for that of SNDRV_PCM_RATE_8000_48000. This is a bit masked field, so you can add whatever rates your harware supports.

My hardware driver describes this data as unsigned 8-bit format (it's actually signed 3-bit g723-24, but ALSA doesn't support that, so I fake it). Most common PCM data is signed 16-bit little-endian (S16_LE). You would use whatever your hardware supplies, which can be more than one type. Since the format is a bit mask, you can define multiple data formats.

Lastly, the info field describes some middle layer features that your hardware/driver supports. What I have here is the base for what most drivers will supply. See the ALSA docs for more details. For example, if your hardware has stereo (or multiple channels) but it does not interleave these channels together, you would not have the interleave flag.

Next post will give us some handler callbacks. It will likely be split into two posts.

<< Prev | Next >>

Writing an ALSA driver: Setting up capture

Now that we have an ALSA card initialized and registered with the middle layer we can move on to describing to ALSA our capture device. Unfortunately for anyone wishing to do playback, I will not be covering that since my device driver only provides for capture. If I end up implementing the playback feature, I will make an additional post.

So let's get started. ALSA provides a PCM API in its middle layer. We will be making use of this to register a single PCM capture device that will have a number of subdevices depending on the low level hardware I have. NOTE: All of the initialization below must be done just before the call to snd_card_register() in the last posting.

struct snd_pcm *pcm;
ret = snd_pcm_new(card, card->driver, 0, 0, nr_subdevs,
                  &pcm);
if (ret < 0)
        return ret;

In the above code we allocate a new PCM structure. We pass the card we allocated beforehand. The second argument is a name for the PCM device, which I have just conveniently set to the same name as the driver. It can be whatever you like. The third argument is the PCM device number. Since I am only allocating one, it's set to 0.

The third and fourth arguments are the number of playback and capture streams associated with this device. For my purpose, playback is 0 and capture is the number I have detected that the card supports (4, 8 or 16).

The last argument is where ALSA allocates the PCM device. It will associate any memory for this with the card, so when we later call snd_card_free(), it will cleanup our PCM device(s) as well.

Next we must associate the handlers for capturing sound data from our hardware. We have a struct defined as such:

static struct snd_pcm_ops my_pcm_ops = {
        .open      = my_pcm_open,
        .close     = my_pcm_close,
        .ioctl     = snd_pcm_lib_ioctl,
        .hw_params = my_hw_params,
        .hw_free   = my_hw_free,
        .prepare   = my_pcm_prepare,
        .trigger   = my_pcm_trigger,
        .pointer   = my_pcm_pointer,
        .copy      = my_pcm_copy,
};

I will go into the details of how to define these handlers in the next post, but for now we just want to let the PCM middle layer know to use them:

snd_pcm_set_ops(pcm, SNDRV_PCM_STREAM_CAPTURE,
                &my_pcm_ops);
pcm->private_data = mydev;
pcm->info_flags = 0;
strcpy(pcm->name, card->shortname);

Here, we first set the capture handlers for this PCM device to the one we defined above. Afterwards, we also set some basic info for the PCM device such as adding our main device as part of the private data (so that we can retrieve it more easily in the handler callbacks).

Now that we've made the device, we want to initialize the memory management associated with the PCM middle layer. ALSA provides some basic memory handling routines for various functions. We want to make use of it since it allows us to reduce the amount of code we write and makes working with userspace that much easier.

ret = snd_pcm_lib_preallocate_pages_for_all(pcm,
                     SNDRV_DMA_TYPE_CONTINUOUS,
                     snd_dma_continuous_data(GFP_KERNEL),
                     MAX_BUFFER, MAX_BUFFER);
if (ret < 0)
        return ret;

The MAX_BUFFER is something we've defined earlier and will be discussed further in the next post. Simply put, it's the maximum size of the buffer in the hardware (the maximum size of data that userspace can request at one time without waiting on the hardware to produce more data).

We are using the simple continuous buffer type here. Your hardware may support direct DMA into the buffers, and as such you would use something like snd_dma_dev() along with your PCI device to initialize this. I'm using standard buffers because my hardware will require me to handle moving data around manually.

Next post we'll actually define the hardware and the handler callbacks.

<< Prev | Next >>

Writing an ALSA driver: The basics

In my last post I described a bit of hardware that I am writing an ALSA driver for. In this installment, I'll dig a little deeper into the base driver. I wont go into the details of the module and PCI initialization that was already present in my driver (I developed the core and v4l2 components first, so all of that is taken care of).

So first off I needed to register with ALSA that we actually have a sound card. This bit is easy, and looks like this:

struct snd_card *card;
ret = snd_card_create(SNDRV_DEFAULT_IDX1, "MySoundCard",
                      THIS_MODULE, 0, &card);
if (ret < 0)
        return ret;

This asks ALSA to allocate a new sound card with the name "MySoundCard". This is also the name that appears in /proc/asound/ as a symlink to the card ID (e.g. "card0"). In my particular instance I actually name the card with an ID number, so it ends up being "MySoundCard0". This is because I can, and typically do, have more than one installed at a time for this type of device. I notice some other sound drivers do not do this, probably because they don't expect more than one to be installed at a time (think HDA, which is usually embedded on the motherboard, and so wont have two or more inserted into a PCIe slot). Next, we set some of the properties of this new card.

strcpy(card->driver, "my_driver");
strcpy(card->shortname, "MySoundCard Audio");
sprintf(card->longname, "%s on %s IRQ %d", card->shortname,
        pci_name(pci_dev), pci_dev->irq);
snd_card_set_dev(card, &pci_dev->dev);

Here, we've assigned the name of the driver that handles this card, which is typically the same as the actual name of your driver. Next is a short description of the hardware, followed by a longer description. Most drivers seem to set the long description to something containing the PCI info. If you have some other bus, then the convention would follow to use information from that particular bus. Finally, set the parent device associated with the card. Again, since this is a PCI device, I set it to that.

Now to set this card up in ALSA along with a decent description of how the hardware works. We add the next bit of code to do this:

static struct snd_device_ops ops = { NULL };
ret = snd_device_new(card, SNDRV_DEV_LOWLEVEL, mydev, &ops);
if (ret < 0)
        return ret;

We're basically telling ALSA to create a new card that is a low level sound driver. The mydev argument is passed as the private data that is associated with this device, for your convenience. We leave the ops structure as a no-op here for now.

Lastly, to complete the registration with ALSA:

if ((ret = snd_card_register(card)) < 0)
        return ret;

ALSA now knows about this card, and lists it in /proc/asound/ among other places such as /sys. We still haven't told ALSA about the interfaces associated with this card (capture/playback). This will be discussed in the next installment. One last thing, when you cleanup your device/driver, you must do so through ALSA as well, like this:

snd_card_free(card);

This will cleanup all items associated with this card, including any devices that we will register later.

<< Prev | Next >>

Writing an ALSA driver

Over the past week I've been writing an ALSA driver for an MPEG-4 capture board (4/8/16 channel). What I discovered is there are not many good documents on the basics of writing a simple ALSA driver. So I wanted to share my experience in the hopes that it would help others.

My driver needed to be pretty simple. The encoder produced 8Khz mono G.723-24 ADPCM. So you can avoid the wikepedia trip, that's 3-bits per sample, or 24000 bits per second. The card produced this at a rate of 128 samples per interrupt (48 bytes) for every channel available (you cannot disable each channel).

The card delivered this data in a 32kbyte buffer, split into 32 pages. Each page was written as 48*20 channels, which took up 960 bytes of the 1024 byte page (it could do up to this number, but for my purposes I was only using 4, 8 or 16 channels of encoded data depending on the capabilities of the card).

Now, let's set aside the fact that ALSA does not have a format spec for G.723-24, so my usage entails dumping out the 48 bytes to userspace as unsigned 8-bit PCM (and my userspace application handles the G.723-24 decoding, knowing that it is getting this data).

First, where to start in ALSA. I had to decide how to expose these capture interfaces. I could have exposed a capture device for each channel, but instead I chose to expose one capture interface with a subdevice for each channel. This made programming a bit easier, gave a better overview of the devices as perceived by ALSA, and kept /dev/snd/ less cluttered (especially when you had multiple 16-channel cards installed). It also made programming userspace easier since it kept channels hierarchically under the card/device.

Next post, I'll discuss how the initial ALSA driver is setup and exposed to userspace.

Next >>