Life, Technology, and Meteorology

Category: Coding (Page 2 of 3)

Using Compressed Textures in OpenGL

I’m not sure if it’s just me, but for some reason OpenGL coding involves a lot of trial and error before getting a feature such as lighting, blending, or texture mapping to work correctly. The past few days I have been working on adding texture compression to my OpenGL map test project. Ultimately, this code will be merged with the rest of the Seasonality source tree, and it’s going to look pretty cool.

Most OpenGL developers will use regular images and possibly compress them when loading them as a texture on the GPU. This is fairly straightforward, and just involves changing one line of code when loading the texture. Note that this is a huge gain when it comes to graphics memory savings, as I was using about 128MB of VRAM when texture compression was disabled and only around 30MB with compression enabled. I wanted to accomplish something a bit more difficult though. I’m going to be using several thousand textures, so I would like to have OpenGL compress them the first time Seasonality is launched, and then save the compressed images back to disk so concurrent launches will not require the re-compression of the imagery.

The problem I ran into was not enough developers are using this technique to speed up their application, so sample code was scarce. I found some in a book I bought awhile back called “More OpenGL Game Programming,” but the code was written for Windows, and it didn’t work on Mac OS X. So I dove deep into the OpenGL API reference and hacked my way through it. The resulting code is a simplification of the method I’m using. It should integrate with your OpenGL application, but I can’t guaranty this completely because it is excerpted from my project. If you’re having a problem integrating it though, post a comment or send me an email.

First, we have some code that will check for a compressed texture file on disk. If the compressed file doesn’t exist, then we are being launched for the first time and should create a compressed texture file.

- (bool) setupGLImageName:(NSString *)imageName
         toTextureNumber:(unsigned int)textureNumber
   GLint width, height, size;
   GLenum compressedFormat;
   GLubyte *pData = NULL;

   // Attempt to load the compressed texture data.
   if (pData = LoadCompressedImage("/path/to/compressed/image", &width, &height,
       &compressedFormat, &size))
      // Compressed texture was found, image bytes are in pData.
      // Bind to this texture number.
      glBindTexture(GL_TEXTURE_2D, textureNumber);

      // Define how to scale the texture.

      // Create the texture from the compressed bytes.
      glCompressedTexImage2D(GL_TEXTURE_2D, 0, compressedFormat,
                             width, height, 0, size, pData);

      // Define your texture edge handling, here I'm clamping.
      glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_R, GL_CLAMP);
      // Free the buffer (allocated in LoadCompressedImage)
      return YES;
   else {
      // A compressed texture doesn't exist yet, run the standard texture code.
      NSImage *baseImage = [NSImage imageNamed:imageName];
      return [self setupGLImage:baseImage toTextureNumber:textureNumber];

Next is the code to load a standard texture. Here we get the bitmap image rep and compress the texture to the GPU. Next we’ll grab the compressed texture and write it to disk.

- (bool) setupGLImage:(NSImage *)image
         toTextureNumber:(unsigned int)textureNumber
   NSData *imageData = [image TIFFRepresentation];
   NSBitmapImageRep *rep = [[NSBitmapImageRep alloc] initWithData:imageData];
   // Add your own error checking here.

   NSSize size = [rep size];
   // Again, more error checking.  Here we aren't using
   // MIPMAPs, so make sure your dimensions are a power of 2.

   int bpp = [rep bitsPerPixel];

   // Bind to the texture number.
   glBindTexture(GL_TEXTURE_2D, textureNumber);
   glPixelStorei(GL_UNPACK_ALIGNMENT, 1);

   // Define how to scale the texture.

   // Figure out what our image format is (alpha?)
   GLenum format, internalFormat;
   if (bpp == 24) {
      format = GL_RGB;
      internalFormat = GL_COMPRESSED_RGB_S3TC_DXT1_EXT;
   else if (bpp == 32) {
      format = GL_RGBA;
      internalFormat = GL_COMPRESSED_RGBA_S3TC_DXT5_EXT;

   // Read in and compress the texture.
   glTexImage2D(GL_TEXTURE_2D, 0, internalFormat,
                size.width, size.height, 0,
                format, GL_UNSIGNED_BYTE, [rep bitmapData]);

   // If our compressed size is reasonable, write the compressed image to disk.
   GLint compressedSize;
   glGetTexLevelParameteriv(GL_TEXTURE_2D, 0,
   if ((compressedSize > 0) && (compressedSize < 100000000)) {
      // Allocate a buffer to read back the compressed texture.
      GLubyte *compressedBytes = malloc(sizeof(GLubyte) * compressedSize);

      // Read back the compressed texture.
      glGetCompressedTexImage(GL_TEXTURE_2D, 0, compressedBytes);

      // Save the texture to a file.
      SaveCompressedImage("/path/to/compressed/image", size.width, size.height,
                          internalFormat, compressedSize, compressedBytes);

      // Free our buffer.

   // Define your texture edge handling, again here I'm clamping.

   // Release the bitmap image rep.
   [rep release];

   return YES;

Finally we have a few functions to write the file to disk and read it from the disk. These functions were pulled almost verbatim from the OpenGL book. In the first code block above we called LoadCompressedImage to read the texture data from the disk. In the second code block, we called SaveCompressedImage to save the texture to disk. Nothing really special is going on here. We write some parameters to the head of the file, so when we go to read it back in we have the details. Bytes 0-3 of the file are the image width, 4-7 is the image height, 8-11 is the format (GL_COMPRESSED_RGB_S3TC_DXT1_EXT or GL_COMPRESSED_RGBA_S3TC_DXT5_EXT), 12-15 is the size of the image data in bytes, and bytes 16+ are the image data.

void SaveCompressedImage(const char *path, GLint width, GLint height,
                         GLenum compressedFormat, GLint size, GLubyte *pData)
   FILE *pFile = fopen(path, "wb");
   if (!pFile)

GLuint info[4];

info[0] = width;
info[1] = height;
info[2] = compressedFormat;
info[3] = size;

fwrite(info, 4, 4, pFile);
fwrite(pData, size, 1, pFile);

GLubyte * LoadCompressedImage(const char *path, GLint *width, GLint *height,
GLenum *compressedFormat, GLint *size)
FILE *pFile = fopen(path, “rb”);
if (!pFile)
return 0;
GLuint info[4];

fread(info, 4, 4, pFile);
*width = info[0];
*height = info[1];
*compressedFormat = info[2];
*size = info[3];

GLubyte *pData = malloc(*size);
fread(pData, *size, 1, pFile);
return pData;
// Free pData when done…

Hopefully this will save someone development time in the future. If you catch any errors, let me know.

Summer Update

It’s been almost two months since I’ve posted here, so to avoid the risk of this blog becoming a dinosaur, I thought I would post an update.

Katrina and I returned from our 6-7 week road trip in the beginning of July. We drove out to California at the end of May, and stayed with family for several weeks (also hitting WWDC, of course). On the way out there, we took the northern route, hitting Mt. Rushmore, Yellowstone, the Tetons, and the Salt Flats. On the way back, we started in Santa Monica and drove Route 66 all the way through St. Louis, taking the freeway the rest of the way home after running out of time. We’ll have to drive the rest of Route 66 from St. Louis to Chicago sometime soon. Overall, it was quite a trip. Watch my Flickr stream for photos of the trip.

Been working on finishing up DynDNS Updater 2.0, which will hopefully be ready soon. The app is looking pretty good. A lot of smaller details have been improved upon since beta 5, that collectively improve the application quite a bit.

Trying to spend some time working on Seasonality’s international forecast as well. I’ll post more on this at a later point in time, but I’ve created some cool imagery and animations that I’ll be using to tweak the forecast generator to make it more accurate.

C4 is coming up this weekend! I’ll be taking off for Chicago tomorrow for a weekend of Indie fun. I’ll be showing an entry for the Iron Coder Live contest…which reminds me I still need to fix a bug or two there. Should be a blast. I’ll most likely be keeping my Twitter feed up to date more than posting here about stuff.

Speaking of Twitter, I started Twittering (is that a word?) a few months ago, and I’m hooked. If you don’t know, Twitter is a place to post Tweets, which are short bits of text (no longer than 160 characters), usually telling others what you’re up to. My first thought was how much time I would be wasting by doing this, but the whole idea is that posting a Tweet is supposed to be really quick. It provides some nice breaks throughout the day, and the community building around the site is pretty amazing. Check out my Twitter page, and if you’re interested, sign up and start using Twitter yourself.

That’s all folks…

What's WWDC??

End-users often ask me what exactly happens at WWDC, or any other development conference for that matter. The question can catch me off-guard. I’ll stumble around for a minute or two as I struggle trying to explain exactly what I get out of a conference like C4 or WWDC. Rob Griffiths from Macworld is attending WWDC this week, and wrote an article on MacCentral to explain just that…

The more immediate connection to Apple employees is certainly one of the biggest advantages of attending WWDC, and I have taken advantage of this a few times this week. However, if I had to choose one key reason to attend a developer conference like this, it would be to have the opportunity to connect and share ideas with other indie developers. Arguably the most innovative applications come from development shops with fewer than 5 developers. These are the applications on the leading edge. Being an indie developer gives us flexibility to choose the latest technologies, and WWDC is a great place to share ideas and talk about how we handle business, marketing, and customer support.

Last night, while listening to Ozomatli perform live at the WWDC San Francisco Bash, I had the pleasure of talking with Luis de la Rosa, Brian Cooke, Tom Harrington, and Lemont Washington about such topics. Other than Luis, I hadn’t spoken to any of these developers much, but it was great extending some connections and talking about Leopard technologies and their effects on our own applications.

Of course a big attraction of WWDC is learning the new APIs and improving our development skills, but that’s just part of the picture. Between the instructional sessions, access to Apple engineers, and sharing ideas with other developers, WWDC is an awesome developer conference. It’s definitely an event I wouldn’t want to miss.

WWDC 2007

I’m typing this from a fresh developer build of Leopard on my MacBook Pro. I’d have to say that this build is much more stable than the last couple of releases.

If you’re a die-hard Mac user (or even if you aren’t), you have probably heard of the various announcements Apple made yesterday. I thought I would run through the most notable announcements and add some commentary. There were three main foci of the keynote: Leopard, Safari for Windows, and the iPhone.


Leopard consumed the majority of the keynote. Steve Jobs went into detail on just 10 of the 300 new features in Leopard. To me, the UI changes were the most significant. Steve went over topics like Time Machine, Spaces, Boot Camp, and 64-bit support, but all of these were talked about at WWDC 2006. The UI changes are fairly substantial, and I imagine there will be quite a bit of commentary regarding these changes across the blogosphere.

The first thing users will notice is the change in the Desktop, which includes the system icons, menubar, and the Dock. The Dock redesign is simply stunning. It’s a very polished, 3-dimensional implementation. The reflections are perfect, and about the only adjustment I would make is with the application shadows, which are actually above the icons. I understand why it was designed this way (as a visual separator of the icon from background content), but it doesn’t make visual sense. Otherwise, I dig it.

Likewise, stacks is a great new feature that should have been implemented by Apple 10 years ago when they originally patented the idea. This is going to make my downloads directory much more manageable. It’s also very useful for traversing directories. When you have a stack of directories, if one of those directories is clicked on, the stack updates itself with the contents of the child directory. This will be a great way to access development project files I’m working on.

However, it’s not all perfect… The folder icons and menubar both need some improvement. To begin with, the icons do not have enough detail. They look like blue blobs from a distance. This is especially noticeable when it comes to “special” directories like Music and Movies. The categorical aspect of the icon is all but distinguishing. I believe all the folder icons need to be made more distinct.

The menubar changes are probably the worst part of the Leopard redesign. The opacity of the entire menubar is around 50%, which really makes the menu titles difficult to read depending on the desktop picture you choose. When a menu is selected, it’s transparency is normal, which is good, but it looks ridiculous under the translucent menubar. At the very least, the menubar needs to fade in to 100% opaque on mouseover. I expect there will be dozens of 3rd party utility hacks to get around this issue, if the menubar makes it as-is into the final release of Leopard.

Update (6/16): That didn’t take long… Here’s the first hack now.

The nicest change about the Leopard desktop is the look of application windows. Finally, Apple is back to a single window layout–merging the standard, brushed metal, and unified layouts into a single, standard window. While it is a bit dark, overall the design is pretty slick. The important point here is that now developers don’t have to choose what kind of base interface to use in their applications. Leopard windows have a single look, and now UI designers can match their icons, views, and controls to that window layout.

Quick look is a nice new feature in Leopard. I really think this feature has the potential to push into obsolescence. It’s very easy to use to display images, presentations, spreadsheets, PDF and word documents. With a quick change to full screen mode, I think this is going to really improve the way users browse through their documents.

The new Finder will probably be seen as a minor upgrade, but I believe it to be significant. The current Finder contains a lot of legacy Carbon code, and Leopard’s Finder should improve performance. It will be difficult to say until I spend more time using it, but I’m hoping threading support is more robust… I hate seeing the spinning beach ball when I get disconnected from a network server, and the new Finder reported fixes this issue. The design also feels cleaner–more enjoyable to use. It’s a subtle, but noticeable improvement. Coverflow also looks neat, but I can only imagine using it in directories filled with images, and even then I would only really use it to scan all the images in general.

The biggest improvements in Leopard will be completely transparent to users… Apple’s going a long way by offering new development technologies behind the scenes, and users will see evidence of these changes in future 3rd party application releases.

Safari for Windows

Safari for Windows is a huge bonus in my eyes. Just from using IE 7 for a few hours, the user interface is a nightmare. Having Safari as an option on my Boot Camp partition here will be very nice. I think Apple is approaching it the right way by marketing towards the web developers with Safari 3’s web debugging features. If web developers are using Safari to develop their applications, then we’re going to see a lot more site compatibility cross-platform. Apple’s plan to market Safari on Windows through their iTunes downloads was vague at best, but if they really do have a million iTunes for Windows downloads every day, that should give them a lot of momentum on entering the Windows browser market.

iPhone “SDK”

Apple’s iPhone announcement was a joke, and far from the “sweet” deal Steve used to describe the paradigm. Developing a web page is not the same as developing an application for the iPhone. Sure, it’s nice that web pages can initiate calls, emails, or physical address searches, but that is not a substitute for developing full iPhone applications. First, while living in a large metropolitan area may keep your iPhone connected 100% of the time, I think that will be far from reality. Without an internet connection, a web page application is useless. Furthermore, applications deserve a position on the user’s home screen. Opening Safari and navigating through bookmarks just to find an application is unacceptable.

In the end, I think John Gruber summarized it best: “It’s great that iPhone seems to have a killer Safari web browser. No doubt there are going to be some terrific web apps targeting iPhone. But there are a ton of great ideas for iPhone software that can’t be done as web apps.”

The Unannouncement

Several media outlets are referring to this years keynote as an unannouncement. While I agree that the keynote failed to show any fancy new hardware or groundbreaking new software, the presentation was packed with details of Apple’s path forward. This is a developer’s conference. The new details shown in Leopard, especially when it comes to user interface design, is important when designing our own applications. In that respect, the keynote showed some nice new UI changes that I think will help bring application consistency back into the picture. Mac OS X is a constantly evolving platform, which I think stems from the semi-frequent release cycle Apple sticks to. Getting a better preview of Leopard’s features gives developers a 4 month headstart on evolving their own applications with the OS, and that’s why this conference is worth it’s cost of admission.

Removing preferences belonging to your app…

I was working on some code to uninstall an application, and was pleasantly surprised to find that Cocoa’s NSUserDefaults class will remove the application’s preferences file from ~/Library/Preferences if you remove all the keys:

NSUserDefaults *defaults = [NSUserDefaults standardUserDefaults];
NSArray *keys = [[defaults dictionaryRepresentation] allKeys];

int i, count = [keys count];
for (i = 0; i < count; i++) {
   [defaults removeObjectForKey:[keys objectAtIndex:i]];

[defaults synchronize];

It’s not often you find uninstall options for OS X applications, but it’s nice Apple’s engineers thought about this case and handle it appropriately.

WWDC Keynote Thoughts

Between all the great sessions here at WWDC yesterday and Buzz’s excellent blogger party last night, I’ve had just about 0 time to blog about anything that has been announced here. The typical news sites have been posting all the details on Mac OS X Leopard that Steve talked about yesterday, but I thought I would add a couple of my own comments on Leopard.

First, though I’m under NDA for a lot of the content here, I’ll just say that Leopard adds a lot of nice features for developers. I would not be surprised to see a lot of applications next year requiring Leopard. I’m sure some Tiger/Panther users will feel a bit left out, but the development time can be collapsed greatly, and these apps will be a lot more polished.

64 bit is a big buzzword around here. It is a big deal…even with 64 bit POSIX available at the UNIX layer in Tiger. That was nice, but it meant that only command-line applications that used straight POSIX libraries would have the ability to run 64 bit. As was mentioned in the keynote, Apple has extended 64 bit support all the way up to the Cocoa and Carbon layers…completing the transition to 64 bit for Mac users. I think this will allow some very high-end scientific applications to provide absolutely beautiful visualization displays without having to write a bunch of extra code to handle 64 bit data processing in a different process on the back-end. I haven’t tried building Seasonality for 64 bit yet, but I suspect that it will provide a slight speed improvement on 64 bit machines because the satellite image is highly accelerated in hardware using the Accelerate framework. 64 bit processors may be able to generate a new satellite image up to twice as fast. I’ll update my blog with performance results on this sometime in the future. changes seem to be aplenty. I haven’t loaded the Leopard preview on my MacBook Pro yet to see just how much has been improved, but already I’m impressed. The templates look to be a good idea, but I can’t see myself using them too often. I’m sure there will be a subset of Mac users that will get a kick out of that though. The notes feature strikes me as a big chunk of bloatware tacked on to Mail. If you need to take notes, there should be another place to do it outside of your inbox. Sure, people spend a lot of time in Mail, and I’m sure a lot of people take notes while reading/responding to email, but that doesn’t mean that notes should be an integrated feature. It seems that a much better solution to write a new system-wide notes application that would let you bring up an interface with a hotkey, type something in, and dismiss it.

Apple still hasn’t updated the Finder. I really hope this is one of the “top secret” features they aren’t releasing until the end. The Finder is something Mac users spend a lot of time using, and the amount of legacy code still in there is pretty staggering. At the very least, the Finder needs to use more threading, but really they should start from scratch and try to implement something that is more efficient. They should also revisit usability. When using the Finder with a modern system with several hundred thousand files, it takes awhile to navigate to where you want to be (Note: this applies to all the current file-system-exploring applications I’ve used on any platform). Spotlight improves this situation somewhat, but it is still a pretty big problem and will only get worse as hard drive capacities skyrocket as they have been in recent years.

Despite these drawbacks, Leopard as a whole is a big improvement. Time Machine and Spaces are greatly welcomed, Core Animation will be a huge win for the usability of Leopard applications and the iChat improvements seem pretty solid.

Developer Benchmarks

A few Mac indie developers like Gus and Luis are running some unofficial developer benchmarks to judge the performance between PowerPC and Intel Macs. The benchmark is compiling Subversion, and it’s a pretty good method of metering performance impact because we spend all day compiling. 🙂

Anyway, I thought I would share the results from my dual 2.5Ghz G5 with 2.5GB of RAM, and to make the mix a little more interesting, I’ve also benchmarked a server here with a single Athlon64 3200+ processor (2Ghz) and 1GB of RAM running Kubuntu Linux.

Dual 2.5Ghz G5:    make -j2    3:22.64 with 174% CPU usage

   make -j4
   3:20.73 with 179% CPU usage

   make -j8
   2:34.94 with 179% CPU usage

   make -j12
   2:37.82 with 178% CPU usage

Athlon64 3200+:    make -j2    3:15.81

   make -j4

   make -j8

I was surprised to see that even though the Athlon box has only a single CPU, the performance is right up there with the dual G5. Also, from Luis’ benchmarks, it looks like my MacBook Pro numbers will be pretty competitive with the G5, though it will be interesting to see how a slower disk effects the benchmarks. Speaking of disk, I should mention that the Athlon box was compiling on a 3 disk RAID 5 partition. Disk usage while compiling should be negligible, but it might skew the numbers a bit.

Now if only Apple would support distributed compiling to any other machine with gcc installed, then I could split my work between the G5, Athlon, and forthcoming MacBook Pro. 🙂

Everything you wanted to know about floats but where too afraid to ask…

There’s a quite interesting blog post over on Ridiculous Fish about how floats are formatted in memory and what kinds of numbers they can represent. This should be required reading for every Computer Science student. I wasn’t taught any of this at UCSB except for the mantissa/exponent way of representing a float and that sometimes an integer stored in a float was not really an integer.


Gus posted a very cool SQLite Objective-C wrapper, complete with sample code. It looks like the methods he has in there are pretty handy. Seasonality uses SQLite as it’s database engine, though at the time I was developing it, the only good wrapper I found was for SQLite 2.8.x, so I went for that. I might end up switching to SQLite 3 in a later version, but we’ll have to see. Since the database files are a completely different format between SQLite 2.8 and SQLite 3.x, there would definitely be some work involved in the upgrade.

I figured this would be a good opportunity to talk more about the database aspects of Seasonality. Seasonality actually uses 2 SQLite databases, one of them is the location database in the application bundle, and the other is stored in the users Application Support directory (~/Library/Application Support/Seasonality). The database in the application bundle is really only used as a read-only database, and it contains all of the locations and their associated data values. When you start typing in a zip code to add a location in Seasonality, the app is actually just running a quick SELECT query from the string you’ve typed so far. There are a few different tables in the location database to associate zip codes to ICAO weather locations and radar locations, but overall it’s a pretty simple database. This is actually a subset of all the data that I’ve mined over the past several months…the master is residing in a PostgreSQL database on one of my servers here, and I wrote a Perl script to select subsets of the data from Postgres and insert them into a new SQLite database to include in Seasonality.

The second database in Application Support is used to store all the saved weather data. There are several fields that are saved for future functionality if needed. Other than the temperature, dewpoint, wind and gust speeds used for the graphs right now, Seasonality saves the air pressure, relative humidity, and the visibility along with the date that data is associated with. This brings up a minor complaint of mine…SQLite is typeless, which means that it treats all values you put in the database as the same type. Integers, Floats, and Dates are all inserted as simple strings. They claim this is a feature, but when it comes to date processing it’s a major pain.

When selecting data to build a weather graph, I want to select data that is between two dates. Unfortunately, if I just use a standard date string like Tue Mar 22 22:45:16 EST 2005, it will choke when I try to do a date comparison because it will look at it as a string and, for example, Fri Mar 25 22:45:16 EST 2005 would be seen as happening before the first date because F in Fri is alphabetically before T in Tue. One way around this is to use a very large integer for the date string: 20050322224516. This works fine, and is the method that Seasonality uses to keep track of dates, but it’s slightly inconvenient because all values are above 2^32, so you need to remember to use 64 bit integers to pass dates in and out of the database.

Back to the second database, that’s all there is at the moment…just that one table with a ton of rows in it. Each row takes maybe 100 bytes, and with an average of around 30 rows a day per a location you end up with a database that is about 1Mb after a year’s time (per location). Not bad at all for the amount of data being saved. Searching it is pretty quick too…my database at the moment has about 11,000 rows in it (data for 5 locations back through the beginning of February), and searching through it takes a fraction of a second. In later versions of Seasonality, I might add a way for users to manage this data a bit better. For instance, exporting the data seems like something that user might want to do, and maybe add some AppleScript handling as well. We’ll see where it goes…


Eric Sink, who has written a lot of different articles about starting a small software business, recently wrote one that hit home for me here at Gaucho Software. The article talks about what he learned after a month of starting his own Micro-ISV, or single-developer software company. His situation is a bit different than mine here, simply because he still has his normal paying job and is programming his Winnable Solitaire on the side, but he still makes a lot of good points in the article.

Speaking of Gaucho Software, I recently configured some discussion forums for the company. I believe it will help me keep in touch with my user population, and will help users keep in touch with other users. There are already a couple of discussions up about XRG, so if you are interested check it out.

« Older posts Newer posts »

© 2022 *Coder Blog

Theme by Anders NorenUp ↑