8. Configuring Libtabula

The default configuration of libtabula is suitable for most purposes, but there are a few things you can change to make it meet special needs.

8.1. The Location of the C API Development Files

Libtabula can be built against several C DBMS APIs such as libsqlite3 and Connector/C. The library build process tries to find these automattically, but sometimes it needs guidance, particularly when you install these files somewhere nonstandard. I’ts a particular problem on Windows, where installation directories are often versioned, so each release of libtabula just has to guess at a few version numbers common at the time of release.

On platforms that use Autoconf[18], the configure script can usually figure out the location of the C API development files by itself[19] It simply tries a bunch of common installation locations until it finds one that works. If your MySQL server was installed in a nonstandard location, you will have to tell the configure script where these files are with some combination of the --with-mysql, --with-mysql-include, and --with-mysql-lib flags. See README-Unix.txt for details.

If you need to modify the build system files, per-platform instructions are in README-Visual-C++.txt, README-MinGW.txt, and README-Mac-OS-X.txt.

8.2. The Maximum Number of Fields Allowed

Libtabula offers two ways to automatically build SQL queries at run time: Template Queries and SSQLS. There’s a limit on the number of fields these mechanisms support, defaulting to 25 fields in the official libtabula packages.[20] The files embodying these limits are lib/querydef.h and lib/ssqls.h, each generated by Perl scripts of the same name but with a .pl extension.

The default querydef.h is small and its size only increases linearly with respect to maximum field count.

ssqls.h is a totally different story. The default 25 field limit makes ssqls.pl generate an ssqls.h over 1 MB. Worse, the field limit to file size relation is quadratic.[21] This has a number of bad effects:

  • Generating header files to support more fields than you actually require is a waste of space and bandwidth.

  • Some compilers have arbitrary limits on the size of macros they’re able to parse. Exceeding these limits usually causes the compiler to misbehave badly, rather than fail gracefully.

  • Because it increases the size of two key files used in building libtabula itself and programs built on it, it increases compile times significantly. One test I did here showed a tripling of compile time from quadrupling the field limit.

  • More than 25 fields in a table is a good sign of a bad database design, most likely a denormalization problem.

The default limits try to mitigate against all of these factors while still being high enough to be useful with most DB designs.

If you’re building libtabula from source on a platform that uses Autoconf, the easiest way to change these limits is at configuration time:

./configure --with-field-limit=50

That causes the configuration script to pass the -f flag to the two Perl scripts named above, overriding the default of 25 fields. Obviously you need a Perl interpreter on the system for this to work, but Perl is usually installed by default on systems libtabula supports via Autoconf.

On all other platforms, you’ll have to give the -f flag to these scripts yourself. This may require installing Perl and putting it in the command path first. Having done that, you can do something like this to raise the limits:

cd lib
perl ssqls.pl -f 50
perl querydef.pl -f 50

Note the need to run these commands within the src subdirectory of the libtabula source tree. (This is done for you automatically on systems where you are able to use the Autoconf method.)

8.3. Buried MySQL C API Headers

It’s common these days on Unixy systems to install the MySQL C API headers in a mysql directory under some common include directory. If the C API headers are in /usr/include/mysql, we say they are “buried” underneath the system’s main include directory, /usr/include. Since the libtabula headers depend on these C API headers, it can be useful for libtabula to know this fact.

When libtabula includes one of the C API headers, it normally does so in the obvious way:

#include <mysql.h>

But, if you define the LIBTABULA_MYSQL_HEADERS_BURIED macro, it switches to this style:

#include <mysql/mysql.h>

In common situations like the /usr/include/mysql one, this simplifies the include path options you pass to your compiler.

8.4. Building on Systems Without Complete C99 Support

Libtabula uses the C99 header stdint.h for portable fixed-size integer typedefs where possible. The C99 extensions aren’t yet officially part of the C++ Standard, so there are still some C++ compilers that don’t offer this header. Libtabula works around the lack of this header where it knows it needs to, but your platform might not be recognized, causing the build to break. If this happens, you can define the LIBTABULA_NO_STDINT_H macro to make libtabula use its best guess for suitable integer types instead of relying on stdint.h.

Libtabula also uses C99’s long long data type where available. Libtabula has workarounds for platforms where this is known not to be available, but if you get errors in common.h about this type, you can define the macro LIBTABULA_NO_LONG_LONGS to make libtabula fall back to portable constructs.

[18] Linux, Solaris, the BSDs, Mac OS X command line (as opposed to the Xcode IDE), Cygwin... Basically, Unix or anything that works like it.

[19] I don’t say “Connector/C” here because the name change generally hasn’t percolated out to Unixy systems. It’s more commonly used on Windows systems, since the separate Connector/C download lets them avoid installing a MySQL server just to get development headers and libraries.

[20] If you’re using a third-party libtabula package, its maintainer may have increased these field counts so the resulting headers more closely approach the size limit of the compiler the package was built with. In that case, you can look at the top of each generated header file to find out how many fields each supports.

[21] The file size equation, for you amateur mathematicians out there, is Nlines = 18.5f2 + 454.5f + 196.4, where f is the field count.