Downtown Doug Brown

I bet you thought I gave up on these lessons! In my last post about microcontrollers, I introduced the idea of an interrupt-safe circular buffer. I briefly mentioned that circular buffers are commonly used in software drivers for peripherals that send and/or receive a lot of data. One example in particular that I gave was a UART (universal asynchronous receiver/transmitter). I’d like to expand further on UARTs in this post.

How they work

First of all, what is a UART? Well, let’s break down the acronym:

• Universal: The device can be configured for different modes/speeds.
• Asynchronous: The communication does not use a separate clock wire for synchronization; it’s all handled only with a single data wire in each direction.
• Receiver: It can receive data…
• Transmitter: …and it can transmit data.

That’s great, but it still doesn’t really explain anything about what the UART does. It’s easier to describe a UART by talking about a computer serial port. If you’re geeky like me, I’m sure you’ve dealt with serial ports before. For example, if you’re a Raspberry Pi aficionado, you might have a USB-to-TTL serial adapter that you hook up to the Pi to access its Linux console. On your computer, you use a program like PuTTY or minicom to open the port for 115,200 baud, 8 data bits, no parity, and 1 stop bit. Then, you can turn on the Pi and see all of the messages it sends at boot. After the operating system is fully booted, you can also enter commands on the console.

If you have done this, you have actually worked with a UART. A UART is this whole interface I just described. You configure it for a baud rate as well as various framing parameters (7-bit data? 8-bit data? 1 stop bit? 2 stop bits? parity bits?). Then you send it bytes. It turns each byte into a sequence of 1s and 0s with a prepended start bit and an appended stop bit or two (there’s also an optional parity bit that can be added to the mix to try to detect errors). Finally, it sends this final sequence of bits onto a transmit pin at the configured bit rate. It also listens on a receive pin for data coming in with that kind of format and parses the serial bits back out into parallel bytes.

On the hardware side of things, let’s go through an example of sending out an uppercase ‘M’ with a UART configured in 8N1 mode. This means there are 8 bits per byte, no parity bit, and 1 stop bit. We start out by telling the UART to send the character ‘M’ which is represented as 0x4D according to an ASCII table. 0x4D in binary is “01001101”. A 1 represents a high state and a 0 represents a low state. The start bit is always a 0 and the stop bit is always a 1.

If you transmitted the character ‘M’ and looked at it on an oscilloscope, this is what you’d see. While the UART is inactive, it keeps the pin at a high state. Thus, the start bit is low so you can detect the start of a transmission. After the stop bit, if there is no more data to be sent, the pin stays high until the next start bit begins.

Hopefully this diagram makes sense. You’ll see that the bits are in reverse order because they are sent least-significant first. The horizontal unit on this diagram is time; the vertical lines are each about 8.7 microseconds apart, assuming the baud rate is set to 115200. I got that by calculating the reciprocal of the bit rate. If the bit rate is 115,200 bits per second, then there are (1/115200) seconds per bit.

The receiver looks for this pattern and interprets the bits between the start and stop bit. That’s really all there is to it! UARTs were very common on computers of the past generations. Earlier computers used a separate chip just for the UARTs, like the 8250 or the 16550A. In fact, many UARTs to this day remain software-compatible with the 16550A. Later on, computers began including a Super I/O chip with integrated 16550A-compatible UARTs.

How to use them

In microcontrollers, you typically find a UART (or multiple UARTs) available as memory-mapped peripherals. Sometimes they use the same interface as the 16550A, and sometimes not. For simplicity I’m going to talk about an AVR’s UART in this lesson instead of the 16550A. The 16550A is complicated because it has a couple of built-in 16-byte FIFO buffers for transmitting and receiving along with a fairly strange interrupt setup because of the FIFOs. The FIFOs are important for normal desktop computers because if your computer had to deal with an interrupt on every character, it would get bogged down handling the interrupts. The FIFO allows interrupts to occur less frequently. On a simple microcontroller like an AVR, handling an interrupt on every character is not a huge problem. Because of the lack of a FIFO, the AVR’s software interface will be easier to understand, which makes it appropriate for this introductory lesson.

I’m going to be using the AT90USB646 as an example. This is the same processor that I use on my Mac ROM SIMM programmer. It has a single USART. Notice the extra S? USART stands for Universal Synchronous/Asynchronous Receiver/Transmitter. It just adds the ability to work in a synchronous mode with an external clock pin. For our purposes, we won’t worry about this. We’ll be using it in asynchronous mode.

The AT90USB646 reference manual says that this chip has a single USART, described in chapter 18. It provides some sample code in both assembly and C for talking to the UART. It’s really not that hard to do.

Setting the baud rate

The baud rate is set with two registers: UBRR1L and UBRR1H. They are the low and high bytes of the full register UBRR1 (which you can also access directly as a 16-bit word, at least in AVR-GCC and AVR Libc). The equation, as per the datasheet, is:

UBRR1 = (oscillator_frequency / (16*baud_rate)) – 1;

So let’s use my SIMM programmer board as an example. It has a 16 MHz crystal, and let’s pretend we’re going for a baud rate of 9600. Thus, I should do:

UBRR1 = (16000000UL / (16*9600UL)) - 1;

Note that I used UL at the end of these constants to ensure they’re being treated as unsigned long integers. Otherwise, the intermediate calculations might get messed up due to only being promoted to 16-bit values. Your development environment will probably provide a define for the clock rate. Plus, it’s usually a good idea to create a #define for your baud rate instead of hardcoding a 9600UL randomly in the code. So it’s actually better to do this:

#define MY_BAUD_RATE 9600UL
UBRR1 = (F_CPU / (16*MY_BAUD_RATE)) - 1;

If you do the math (and integer truncation) yourself, you will discover that this will assign UBRR1 a final value of 103. Mathematically, it would be 103.166666666…, but it will get truncated down to an integer. This also means the baud rate won’t be exactly 9600 baud. If we solve the equation above for baud_rate and put our calculated/truncated UBRR1 value of 103 back in, you’ll see that we don’t get exactly 9600 baud:

baud_rate = oscillator_frequency / (16*(UBRR1 + 1));

baud_rate = 16000000 / (16*(103 + 1));

This gives us a baud rate of 9615. The AVR will actually be transmitting and receiving at 9615 baud. That’s an error of about 0.16%, which is definitely close enough to 9600 baud that it won’t be a problem. Usually you’re in good shape if you’re within a percent or two, and we are much closer than that. Keep in mind that oscillators aren’t perfect either, so UARTs are tolerant of baud rates that aren’t exactly correct.

Configuring the framing

We have to make sure that the USART knows what type of data to send and receive. Should there be 5 data bits? 8 data bits? 1 stop bit? 2 stop bits? Parity? No parity? All of this can be configured in some of the registers in the USART. The registers are called UCSR1A, UCSR1B, and UCSR1C. UCSR is short for “USART Control and Status Register”. These registers contain bits for controlling the setup of the USART as well as some bits that signal status of the USART.

The bits that are important for us to configure in order to get basic functionality are the following:

• U2X1 bit (bit 1) of UCSR1A. This is a double-speed bit that essentially turns the constant “16” in the baud rate calculation formula above into an “8” — we should make sure it’s disabled since we assumed in the baud rate calculation we did earlier that it was in fact 16.
• RXEN1 bit (bit 4) of UCSR1B. This bit enables the receiver so we can receive incoming bytes.
• TXEN1 bit (bit 3) of UCSR1B. This bit enables the transmitter so we can send outgoing bytes.
• UCSZ12 bit (bit 2) of UCSR1B and UCSZ11, UCSZ10 bits (bits 2 and 1) of UCSR1C. These three bits combine together to determine the number of data bits (5 to 9).
  • 0 –> 5 bits
  • 1 –> 6 bits
  • 2 –> 7 bits
  • 3 –> 8 bits (what we want)
  • 4, 5, 6 reserved
  • 7 –> 9 bits
• UMSEL11, UMSEL10 bits (bits 7 and 6) of UCSR1C. These bits determine the mode (synchronous/asynchronous) of the USART.
  • 0 –> asynchronous (what we want)
  • 1 –> synchronous
  • 2 –> reserved
  • 3 –> master SPI (this USART is capable of being an SPI master; I have discussed SPI in the past)
• UPM11, UPM10 bits (bits 5 and 4) of UCSR1C. These bits determine the parity mode (none, even, or odd).
• USBS1 bit (bit 3) of UCSR1C. This bit determines if there is 1 stop bit or 2 stop bits.

So basically, we need to set up these three registers:

UCSR1A = 0; // Not double speed mode, disable address filtering
UCSR1B = 0x18; // Enable RX and TX, high bit of UCSZ1 = 0
UCSR1C = 0x06; // asynchronous, no parity, 1 stop bit, 8 data bits

AVR Libc has some nice macros for the various bit names, so you can also do this with the symbolic names:

UCSR1A = 0;
UCSR1B = (1 << RXEN1) | (1 << TXEN1);
UCSR1C = (3 << UCSZ10);

Transmitting data

After you have configured the USART correctly, it’s very easy to send and receive. To send data, you simply write the character you wish to send into the UDR1 register. Here’s an example of transmitting the character ‘A’:

UDR1 = 'A';

You can only transmit if the hardware has finished transmitting the last character. This is because the USART does not have a transmit FIFO. Because of this, you need to check a status bit to ensure the transmitter is not already busy:

if (UCSR1A & (1 << UDRE1)) {
    UDR1 = 'A';
}

This will only transmit a character if the transmit buffer is empty. Otherwise, it won’t transmit. If you need to make sure the character is transmitted, you can wait until the bit goes high:

while (!(UCSR1A & (1 << UDRE1))) {
}
UDR1 = 'A';

So you do an empty loop until the bit goes high, after which it’s safe to write to the transmit buffer.

Receiving data

Receiving is very, very similar. First, you need to know if a character is ready to be received. The hardware will tell you if a character is waiting to be read by making a bit high in the UCSR1A register. If the bit is high, you can read the character by reading the UDR1 register:

if (UCSR1A & (1 << RXC1)) {
    received_char = UDR1;
    // do something with the received character
}

The receiver actually has a FIFO (first-in, first-out) buffer to hold a couple of received characters in case your program is unable to read the first character received before another one arrives. This is a nice little safety mechanism to have, but you still need to read received characters quickly before the small FIFO fills up. Fancier UARTs have a bigger receive buffer to enable high data rates with less processor utilization.

Example code

Here’s a sample program that will echo back any character that is received. If you type the letter ‘Z’, it will transmit the letter ‘Z’ right back to you.

#include <avr/io.h>

#define MY_BAUD_RATE 9600UL

int main(void) {
    UBRR1 = (F_CPU / (16*MY_BAUD_RATE)) - 1;
    UCSR1A = 0;
    UCSR1B = (1 << RXEN1) | (1 << TXEN1);
    UCSR1C = (3 << UCSZ10);
    
    while (1) {
        while (!(UCSR1A & (1 << RXC1)));
        uint8_t ch = UDR1;
        while (!(UCSR1A & (1 << UDRE1)));
        UDR1 = ch;
    }
}

It’s really not that complicated. It waits until a character is received, and then receives it. Then it waits until the transmitter is empty, and transmits the received character. Note that I put semicolons at the end of the while() loops waiting for the bits. This was done intentionally to save space over giving them an empty loop body, but it can be a little confusing to read, so I wanted to explicitly point that out.

Anyway, this whole thing is easy, right?

Interrupts

There’s a catch. If your program has to do anything other than sit in a loop waiting for a character to arrive over the UART, this whole idea starts to fall apart. If you don’t check for received data often enough, you will run the risk of losing received characters. This could happen if the receive FIFO is filled before you get around to checking the UART again. How do we solve this dilemma?

You may have been wondering why I mentioned circular buffers at the beginning of this post. What do circular buffers have to do with anything I have mentioned so far? Good news: you’ve finally reached the point in this post where everything will start to come together and make sense.

The solution to this problem is to handle the UART with an interrupt. Whenever a character is received, the UART will fire an interrupt. This will cause your main program to temporarily jump to an interrupt handler which reads the received character from the UDR1 register and stores it in a circular buffer. Somewhere in the main loop, you can occasionally check the circular buffer to see if it contains any characters and process them. As long as the buffer is big enough so that you will check it before it fills up, it will solve the problem.

On the other end, you can also create a separate circular buffer for transmitting over the UART. This will allow you to put the data into the circular buffer and then move on to other places in your program while the data is sent in the background by your interrupt handler. Basically, it prevents you from having to sit in a loop and wait for all of your characters to be transmitted before you can move onto other tasks.

This a perfect application for a circular buffer. It’s easy to make a circular buffer interrupt-safe as long as you insert from the main loop and remove from the interrupt handler (or vice versa), and inserting and removing from it are both O(1) operations. This is exactly what we need for both the transmitting and receiving end of things.

The AVR’s USART is really handy because it has what’s known as the UDRE interrupt, which stands for USART data register empty. This interrupt fires anytime the transmit data register is ready for you to write another character. It’s based on the same status bit we checked in the non-interrupt-driven code above. So if the USART is idle, this interrupt will fire. Because of this interrupt, there are no special cases when you first start transmitting characters. When you put a character in the TX buffer, you have to ensure that the UDRE interrupt is enabled. In your UDRE interrupt handler, if the TX buffer is empty, you disable the UDRE interrupt. This logic is very simple compared to other UARTs, which often require special cases when you transmit your first character in order to get the interrupts rolling.

Example code

Here’s some sample code I wrote for interrupt-driven UART functionality. It consists of a header file and source file for the UART driver. You will notice the ring buffer functions which I renamed and borrowed from my post about ring buffers. Many AVR microcontrollers have multiple USARTs. This particular one works with USART1 on the AT90USB646.

uart.h:

#ifndef UART_H_
#define UART_H_

#include <stdbool.h>
#include <stdint.h>

void uart_init(uint32_t baud);
void uart_write_char(char data);
char uart_read_char(void);
bool uart_rx_buffer_empty(void);
bool uart_tx_buffer_empty(void);
bool uart_rx_buffer_full(void);
bool uart_tx_buffer_full(void);

#endif /* UART_H_ */

uart.c:

#include "uart.h"
#include <avr/io.h>
#include <avr/interrupt.h>

#define RING_SIZE   128
typedef uint8_t ring_pos_t;

static volatile ring_pos_t tx_ring_head;
static volatile ring_pos_t tx_ring_tail;
static volatile char tx_ring_data[RING_SIZE];

static volatile ring_pos_t rx_ring_head;
static volatile ring_pos_t rx_ring_tail;
static volatile char rx_ring_data[RING_SIZE];

static int tx_ring_add(char c);
static int tx_ring_remove(void);
static int rx_ring_add(char c);
static int rx_ring_remove(void);

void uart_init(uint32_t baud) {
    // TODO: if you are OK with hardcoding the bit rate,
    // consider using avr-libc's <util/setbaud.h>
    // functionality instead.

    // Set baud rate
    UBRR1 = (F_CPU / (16*baud)) - 1;
    // Enable RX and TX, turn on RX interrupt,
    // turn off data register empty interrupt until needed
    UCSR1B = (1 << RXEN1) | (1 << TXEN1) | (1 << RXCIE1);
    // 8 data bits, 1 stop bit
    UCSR1C = (3 << UCSZ10);

    // Clear out head and tail just in case
    tx_ring_head = 0;
    rx_ring_head = 0;
    tx_ring_tail = 0;
    rx_ring_tail = 0;
}

void uart_write_char(char data) {
    // Wait until there's room in the ring buffer
    while (uart_tx_buffer_full());

    // Add the data to the ring buffer now that there's room
    tx_ring_add(data);

    // Ensure the data register empty interrupt is turned on
    // (it gets turned off automatically when the UART is idle)
    UCSR1B |= (1 << UDRIE1);
}

char uart_read_char(void) {
    // Wait until a character is available to read
    while (uart_rx_buffer_empty());

    // Then return the character
    return (char)rx_ring_remove();
}

bool uart_rx_buffer_empty(void) {
    // If the head and tail are equal, the buffer is empty.
    return (rx_ring_head == rx_ring_tail);
}

bool uart_tx_buffer_empty(void) {
    // If the head and tail are equal, the buffer is empty.
    return (tx_ring_head == tx_ring_tail);
}

bool uart_rx_buffer_full(void) {
    // If the head is one slot behind the tail, the buffer is full.
    return ((rx_ring_head + 1) % RING_SIZE) == rx_ring_tail;
}

bool uart_tx_buffer_full(void) {
    // If the head is one slot behind the tail, the buffer is full.
    return ((tx_ring_head + 1) % RING_SIZE) == tx_ring_tail;
}

static int tx_ring_add(char c) {
    ring_pos_t next_head = (tx_ring_head + 1) % RING_SIZE;
    if (next_head != tx_ring_tail) {
        /* there is room */
        tx_ring_data[tx_ring_head] = c;
        tx_ring_head = next_head;
        return 0;
    } else {
        /* no room left in the buffer */
        return -1;
    }
}

static int tx_ring_remove(void) {
    if (tx_ring_head != tx_ring_tail) {
        int c = tx_ring_data[tx_ring_tail];
        tx_ring_tail = (tx_ring_tail + 1) % RING_SIZE;
        return c;
    } else {
        return -1;
    }
}

static int rx_ring_add(char c) {
    ring_pos_t next_head = (rx_ring_head + 1) % RING_SIZE;
    if (next_head != rx_ring_tail) {
        /* there is room */
        rx_ring_data[rx_ring_head] = c;
        rx_ring_head = next_head;
        return 0;
    } else {
        /* no room left in the buffer */
        return -1;
    }
}

static int rx_ring_remove(void) {
    if (rx_ring_head != rx_ring_tail) {
        int c = rx_ring_data[rx_ring_tail];
        rx_ring_tail = (rx_ring_tail + 1) % RING_SIZE;
        return c;
    } else {
        return -1;
    }
}

ISR(USART1_RX_vect) {
    char data = UDR1;
    rx_ring_add(data);
}

ISR(USART1_UDRE_vect) {
    if (!uart_tx_buffer_empty()) {
        // Send the next character if we have one to send
        UDR1 = (char)tx_ring_remove();
    } else {
        // Turn off the data register empty interrupt if
        // we have nothing left to send
        UCSR1B &= ~(1 << UDRIE1);
    }
}

Finally, here’s a test program that tests out the USART:

#include <avr/interrupt.h>
#include "uart.h"

int main(void) {
    cli(); // disable interrupts
    uart_init(9600);
    sei(); // enable interrupts
    while (1) {
        uart_write_char(uart_read_char());
    }
}

This particular sample program doesn’t really take advantage of the fact that the code is interrupt-driven, but at least it tests the code out. The interrupt-driven aspect of the code shines when you need to send a 50-byte string and don’t want to wait around until it finishes sending. With this code, if you quickly write 50 bytes, the program will continue doing other things while the 50 bytes are sent in the background as interrupts come in. For receiving, you can let your program do many things without worrying about losing a received character. Every time through your main loop, you should process all characters that are waiting in the receive buffer to ensure that it doesn’t fill up.

Other functions

You could easily add other functions for transmitting a string, receiving a full line, etc. It would probably be nice for the transmit and receive functions to have optional parameters to tell them not to block if the buffers aren’t ready. These are all excellent exercises I leave to you, the reader, to implement.

Conclusion

I hope this explains the UART/USART in enough detail to get you started. This is a very important peripheral that is commonly used for interfacing with displays, sensors, and other microcontrollers. If you’re going to use a UART/USART, I would highly recommend that you do it with interrupts as I described above. This is definitely the type of thing you should stick into your toolbox of peripheral drivers and reuse on all of your projects.

Have you ever wanted to be able to call a C++ Qt library from Python? It turns out it’s pretty easy to do. This tutorial explains how to do it in Mac OS X, but it’s slightly out of date. I’d like to provide the world with a way to make it work with Ubuntu 14.04 out of the box. My instructions are based on that tutorial and the PySide Binding Generation Tutorial. Let’s check it out. Note: This mechanism seems to work for wrapping Qt 4 libraries. I don’t think PySide works with Qt 5, at least in Ubuntu 14.04. Keep that in mind. Everything we do will be with Qt 4.

We’re going to create a library called libdougtest. It’s going to contain a QObject subclass called TestClass. We will create a Python wrapper for it called libDougTestPy, which we will eventually be able to refer to as DougTestPy to follow Python’s library naming conventions. Let’s get started.

Install some prerequisites

There are many things you need installed:

sudo apt-get install build-essential qt-sdk python-pyside* libshiboken-dev shiboken pyside-tools libpyside-dev

I got a warning saying I should install a Phonon backend when I did this, so I also installed the package phonon-backend-gstreamer just in case.

Create a project directory

We’re going to start out by creating a directory for this entire project. Let’s call it  dougtest_project:

mkdir dougtest_project
cd dougtest_project

All work we do will be inside of this directory. The reason I’m naming it dougtest_project instead of just dougtest is because I’m going to create another directory inside of it called dougtest for the C++ library itself.

Create libdougtest

Now, let’s create the good old C++ library that we want to wrap. This is pretty easy, since you just follow the standard Qt way of creating a library. For simplicity, I’ve provided a barebones library containing a single class. We are going to create a directory inside of dougtest_project called dougtest which will be the home for this library:

mkdir dougtest

First of all, create the project file:

dougtest/dougtest.pro:

QT += core
TARGET = dougtest
TEMPLATE = lib

HEADERS += testclass.h
SOURCES += testclass.cpp

Now, create the header and source files:

dougtest/testclass.h:

#ifndef TESTCLASS_H
#define TESTCLASS_H

#include <QObject>

class TestClass : public QObject
{
    Q_OBJECT
public:
    TestClass(QObject *parent = 0);
    virtual ~TestClass();
 
    void doSomething();

private:
    int value;
};

#endif

dougtest/testclass.cpp:

#include "testclass.h"

TestClass::TestClass(QObject *parent) :
    QObject(parent),
    value(0)
{
 
}

TestClass::~TestClass()
{
}

void TestClass::doSomething()
{
    value++;
    qDebug("Testing: %d", value);
}

This concludes the creation of the library. Let’s make sure it compiles:

cd dougtest
qmake-qt4
make

If you’re successful, you should see libdougtest.so.1.0.0 with several symlinks. Change back to the main directory:

cd ..

OK, we have successfully created the library that we’re going to wrap. We haven’t seen anything special yet; the fun is coming now.

Create files for shiboken

Next, we’re going to create some files that shiboken will use during the wrapping process. shiboken is a binding generator for creating Python bindings for C++ libraries, which is exactly what we want.

mkdir data

Create the following two files:

data/global.h:

#undef QT_NO_STL
#undef QT_NO_STL_WCHAR

#ifndef NULL
#define NULL 0
#endif

#include <QtCore/QtCore>

#include <pyside_global.h>
#include <testclass.h>

data/typesystem.xml:

<?xml version="1.0"?>
<typesystem package="DougTestPy">
    <load-typesystem name="typesystem_core.xml" generate="no"/>
    <object-type name="TestClass"/>
</typesystem>

These files specify the classes to wrap, and will be passed as parameters to shiboken, as you will see shortly.

Create wrapped library project file

Next, we need to create a Qt project that will generate the wrapper library. Create the directory DougTestPy (inside of the main dougtest_project directory):

mkdir DougTestPy

Create the file DougTestPy/DougTestPy.pro:

TEMPLATE = lib
QT += core

# Need to have access to header files from the library being wrapped.
INCLUDEPATH += ../dougtest

# PySide include paths
QMAKE_CXXFLAGS += $$system(pkg-config --cflags pyside)
INCLUDEPATH += $$system(pkg-config --variable=includedir pyside)/QtCore

# PySide dependencies
LIBS += $$system(pkg-config --libs pyside)

# Link against the library being wrapped
LIBS += -L../bin/ -ldougtest

# Generate the wrapper library in the bin directory
TARGET = ../bin/DougTestPy

# Here are all the sources that go with it
SOURCES += \
    DougTestPy/dougtestpy_module_wrapper.cpp \
    DougTestPy/testclass_wrapper.cpp

This script may be confusing because it refers to source files that we haven’t created yet. This is the naming scheme that shiboken will use when we use it to create the wrapper code.

Create the build script and run it

We are finally ready to create a shell script that will call shiboken and do everything that needs to be done. I hope this isn’t too complicated. It’s very possible that a Makefile would work too, but this is what I have working.

Create the file build.sh inside the main dougtest_project directory:

#!/bin/sh

# Absolute path to this script, e.g. /home/user/bin/foo.sh
SCRIPT=$(readlink -f "$0")
# Absolute path this script is in, e.g. /home/user/bin
SCRIPTPATH=$(dirname "$SCRIPT")

WRAPPER_NAME="DougTestPy"
WRAPPED_NAME="dougtest"

QT_INC=$(pkg-config --variable=includedir QtCore)/..
QTCORE_INC=$(pkg-config --variable=includedir QtCore)
PYSIDE_INC=$(pkg-config --variable=includedir pyside)
QTTYPESYSTEM=$(pkg-config --variable=typesystemdir pyside)

# I'm assuming the wrapped library is built in-place; if not,
# change WRAPPED_BIN_DIR to the location of the final binaries.
WRAPPED_SRC_DIR="${SCRIPTPATH}/${WRAPPED_NAME}"
WRAPPED_BIN_DIR="${SCRIPTPATH}/${WRAPPED_NAME}"

cd $SCRIPTPATH

# Clean up the old build to ensure we start fresh. If we don't do this,
# old stale stuff can be left behind, which can cause crashes in the
# library after you make changes.
rm -rf ./bin ./${WRAPPER_NAME}/Makefile ./${WRAPPER_NAME}/${WRAPPER_NAME}

# Copy the latest C++ library build into the bin directory.
mkdir -p bin
cp -R ${WRAPPED_BIN_DIR}/lib${WRAPPED_NAME}* ./bin/

# Now work inside the wrapper directory
cd $WRAPPER_NAME

# Create the wrappers for the library
shiboken ../data/global.h \
    --include-paths=$WRAPPED_SRC_DIR:$QT_INC:$QTCORE_INC:$PYSIDE_INC \
    --typesystem-paths=../data:$QTTYPESYSTEM \
    --output-directory=. \
    --avoid-protected-hack \
    ../data/typesystem.xml
if [ $? -ne 0 ]; then
    echo "shiboken returned error. Something is shibroken :-("
    exit 1
fi

# Build the python wrapper library
qmake-qt4
if [ $? -ne 0 ]; then
    echo "qmake-qt4 returned error."
    exit 1
fi

make
if [ $? -ne 0 ]; then
    echo "make returned error."
    exit 1
fi

# Back into the directory containing the script
cd ..

# Make meaningful symlink so Python can use the generated library
rm -rf ./bin/${WRAPPER_NAME}.so
ln -s lib${WRAPPER_NAME}.so ./bin/${WRAPPER_NAME}.so

OK, you’re ready to run this build script:

sh build.sh

Assuming everything succeeds, you should end up with a bin directory with the following contents:

• DougTestPy.so (a symlink to libDougTestPy.so)
• libDougTestPy.so, libDougTestPy.so.1, libDougTestPy.so.1.0, and libDougTestPy.so.1.0.0
• libdougtest.so, libdougtest.so.1, libdougtest.so.1.0, and libdougtest.so.1.0.0

The symlink has to be created because the build process generates a library that begins with “lib”, but Python expects the library to not be prefixed with “lib”. Maybe there is a way to fix it so that qmake doesn’t add the “lib” prefix, but I couldn’t figure out how after a quick search, and this did the trick, so I let it go.

Test the library

We’re ready to test it out. Create the file test.py in the main dougtest_project directory:

import sys
sys.path.append("bin")
from PySide.QtCore import *
from DougTestPy import *

# Create a TestClass instance
tc = TestClass()

print "Created test class"

# Test the TestClass
tc.doSomething()

print "Called test class function"

tc.doSomething()

print "Called test class function again"

Now, run it. Note that I had to add the “bin” directory to Python’s path in order to use the library. We’re not done with library trickery yet though: libDougTestPy.so depends on libdougtest.so. So libdougtest.so needs to be in the system library path. The easiest way to do this temporarily is to add the bin directory to LD_LIBRARY_PATH:

export LD_LIBRARY_PATH=bin/

For a more permanent solution, you could put libdougtest.so* into /usr/local/lib and run “sudo ldconfig” to eliminate the need for that hack.

Now, run the test script:

python test.py

You should get this output:

Created test class
Testing: 1
Called test class function
Testing: 2
Called test class function again

Conclusion

I hope this was helpful. It’s really not that bad; it’s just that it requires a lot of manual setup. It would be nice if there was something that could automatically generate all the junk we had to make by hand. Maybe such a tool already exists and I’m wasting my time. Either way, I think this is interesting. Thanks again to the people who wrote the tutorials I linked above.

Note that if you use libraries other than QtCore, you will need to add them into the appropriate places in data/global.h, data/typesystem.xml, DougTestPy/DougTestPy.pro, and build.sh. Just see where the word “core” or “Core” appears, and add your extra libraries such as QtGui and QtXml in the same format.

I’ve already gotten my Willem programmer working in Windows 7 with a custom DLL, so I figured why not do the same in Wine under Linux? I meant to make directions for this earlier, but the last time I looked at this, Wine had a bug that made it really annoying to build 32-bit DLLs on 64-bit Linux. That’s all fixed now, so here we go. These instructions are tested with Ubuntu 14.04 with Wine 1.6. Some older versions of Wine had problems, so this is all I can test with.

Warning: This DLL is slow, and these instructions are fairly complicated. I don’t really want to supply prebuilt binaries, and it requires some special work that might have to be repeated when new versions of Wine are installed due to updates. I’m also too lazy to make it configurable, so you need to change a #define in the code if your parallel port is at an I/O address other than 0x378. If you’re still feeling brave, here’s what I have:

Get your parallel port address

First of all, figure out your parallel port address. Type the following command:

cat /proc/ioports | grep parport

You should see something like this:

  0378-037a : parport0

In this case, the output means my parallel port is at 0x378. Remember this address; we will customize it when we compile the DLL.

Download the DLL source code

Download the DLL source code here:

io_dll_willem_linux.tar.gz

Extract it. If your parallel port is at an address other than 0x378, open up io_main.c and update the LPT_BASE_ADDR define near the top of the code to match your parallel port address.

Install prerequisites

These packages will probably be necessary to make everything work. I hope I got all of them, but I may have forgotten something. If I forgot something, I apologize. Let me know in the comments.

sudo apt-get install build-essential wine1.6-dev gcc-multilib

Build the DLL

Inside the DLL source directory, type:

make

Assuming everything goes without error, you should find the file io.dll.so which is the DLL we will be using.

Install the DLL

Remove any io.dll from your Wine installation. It’s useless inside of Wine anyway, so there’s no need for it. Make sure there’s no io.dll file next to EpromM51.exe either. I’m trying to make sure that Wine doesn’t look anywhere else for this DLL. Now, we’re going to put the io.dll.so file in a place where Wine can find it:

sudo cp io.dll.so /usr/lib/i386-linux-gnu/wine/

I messed around, but couldn’t get this to work without putting it right in Wine’s library directory. There might be a better way to do this, but this was the best I could find.

Even if you’re on a 64-bit system, you’ll find that this DLL gets compiled as a 32-bit DLL. So /usr/lib/i386-linux-gnu/wine is always the correct location for this library, regardless of whether your system is 32-bit or 64-bit.

Set up permissions

Here’s the weird part. Raw port access doesn’t work unless you’re root, or you allow the program to have capabilities to access I/O ports. Running things as root is a bad idea, so let’s give the program capabilities to access the I/O ports. Warning: This could pose a potential security threat to your system. We’re actually going to be giving all Wine programs access to the I/O ports.

Here’s the magic command that allows Wine programs to use raw I/O:

sudo setcap cap_sys_rawio=ep /usr/bin/wine-preloader

You may or may not have to re-run this command every time you update Wine. I just don’t know. Anybody else have a better solution for direct port access? I hate having to mess with wine-preloader, but I don’t want to create a separate daemon or whatever due to the added latency.

All done

If you’re followed all these directions, it’s very possible that your Willem programmer is all ready to go now. Give it a shot. I hope this helps someone out there.

Boring details for developers

For anyone who’s interested in how I did this, I created this DLL by taking the original io.dll, fixing its header file so it doesn’t use weird typedefs that confuse winedump, and then running this command:

winedump spec io.dll -I . -t -C

After that, I ran this command:

winemaker . --nosource-fix --dll --nomfc -D__WINESRC__ --wine32

This gave me everything I needed to get this to work. I also had to fix up the generated code–for example, I commented out the line: #include “config.h” in io_main.c.