We’ve not seen what they’ve done but what we can say is that they don’t have 10,000 people in R&D in the vision category.

They don’t have the best scaling engine in the world and they don’t have world renowned picture quality that has been awarded more than anyone else.

TVs are ultimately about picture quality. Ultimately. How smart they are…great, but let’s face it that’s a secondary consideration. The ultimate is about picture quality and there is no way that anyone, new or old, can come along this year or next year and beat us on picture quality.

He makes a good point: TVs are about picture quality. The thing is, I’ve never seen another brand of computer monitor (perhaps high-end NEC displays, but those are on a different pricing level) hold a candle to the picture quality you get from any of Apple’s modern LCDs. Picture quality just isn’t going to be an issue if Apple ever ships an iTV.

The quote above reminded me a lot of a statement Palm’s then-CEO Ed Colligan made back in 2006 about the rumored iPhone:

“We’ve learned and struggled for a few years here figuring out how to make a decent phone. PC guys are not going to just figure this out. They’re not going to just walk in.

We all know how that turned out…

Apps for Water gives app developers a chance to donate one day of their sales to a very worthy cause: providing clean water for developing nations. It seems like a simple thing, turn on a faucet and clean water pours out. Unfortunately, there are almost 1 billion people in the world who don’t have a safe, clean water source. This isn’t just a minor inconvenience. It turns out, the lack of safe water is one of the leading causes of death worldwide. Over 2,000,000 people die each year from diseases stemming from the lack of clean water.

This is where organizations like charity: water come in. The goal of these organizations is to provide safe water for communities in developing nations. Believe it or not, it only costs about $20 to give a new person access to clean water. And because of a collection of private donors that pay for the day-to-day operations of charity: water, every dime that is donated by 3rd parties to their organization goes directly toward water projects.

So here’s the deal. Gaucho Software will be donating 100% of the proceeds from app sales on December 20th, 2011 to charity: water. Help fund a water project, get an app for free. Furthermore, as an additional incentive to donate, Seasonality Core and Seasonality Go will be on sale that day for 20% off.

A call to app developers…

But the hope is that my offer will be just the tip of the iceberg. I would like to formally invite all other developers of Mac and iOS apps to participate in this campaign. We’re setting up a new website¹ at AppsForWater.com to list all the apps that will be giving discounts and donating their proceeds that day to charity: water. Then we’ll get the word out, and tell everyone about how they can give clean water to communities in need.

If you are a developer who is interested in participating, please send me an email or ping me on Twitter. I will do my best to answer any questions and will let you know what materials I need to have your apps listed.

It’s been shown time and time again that the Mac and iOS community can make a big difference in the world. Please help us do it again.

¹ I want to send a special thanks to François Joseph de Kermadec and the rest of the team at Trumper, Forsyth & Kline for so generously donating their time to develop the website for this project.

I’m still in the process of working around the problem, but I wanted to make sure that any work-around would not affect the other 99% of my users who don’t have this graphics card. So I set out to try and find a method of detecting which graphics cards are installed in a user’s Mac. You can use the system_profiler terminal command to do this:

system_profiler SPDisplaysDataType

But running an external process from within the app is slow, and it can be difficult to parse the data reliably. Plus, if the system_profiler command goes away, the application code won’t work. I continued looking…

Eventually, I found that I might be able to get this information from IOKit. If you run the command ioreg -l, you’ll get a lengthy tree of hardware present in your Mac. I’ve used IOKit in my code before, so I figured I would try to do that again. Here is the solution I came up with:

// Check the PCI devices for video cards.
CFMutableDictionaryRef match_dictionary = IOServiceMatching("IOPCIDevice");

// Create a iterator to go through the found devices.
io_iterator_t entry_iterator;
if (IOServiceGetMatchingServices(kIOMasterPortDefault,
                                 match_dictionary,
                                 &entry_iterator) == kIOReturnSuccess)
{
  // Actually iterate through the found devices.
  io_registry_entry_t serviceObject;
  while ((serviceObject = IOIteratorNext(entry_iterator))) {
    // Put this services object into a dictionary object.
    CFMutableDictionaryRef serviceDictionary;
    if (IORegistryEntryCreateCFProperties(serviceObject,
                                          &serviceDictionary,
                                          kCFAllocatorDefault,
                                          kNilOptions) != kIOReturnSuccess)
    {
      // Failed to create a service dictionary, release and go on.
      IOObjectRelease(serviceObject);
      continue;
    }

    // If this is a GPU listing, it will have a "model" key
    // that points to a CFDataRef.
    const void *model = CFDictionaryGetValue(serviceDictionary, @"model");
    if (model != nil) {
      if (CFGetTypeID(model) == CFDataGetTypeID()) {
        // Create a string from the CFDataRef.
        NSString *s = [[NSString alloc] initWithData:(NSData *)model
                                            encoding:NSASCIIStringEncoding];
        NSLog(@"Found GPU: %@", s);
        [s release];
      }
    }

    // Release the dictionary created by IORegistryEntryCreateCFProperties.
    CFRelease(serviceDictionary);
  }
}

11″ MacBook Air (Summer 2011)
1.8Ghz Core i7
256GB SSD (Samsung model)

First the baseline. Before enabling File Vault, I did a quick test of writing a 10GB file and then reading it back.

Blade:~ mike$ time dd if=/dev/zero of=test bs=1073741824 count=10
10+0 records in
10+0 records out
10737418240 bytes transferred in 41.938686 secs (256026577 bytes/sec)
real 0m42.021s
user 0m0.005s
sys 0m6.827s
Blade:~ mike$ time dd if=test of=/dev/null bs=1073741824 count=10
10+0 records in
10+0 records out
10737418240 bytes transferred in 37.969485 secs (282790726 bytes/sec)
real 0m38.055s
user 0m0.005s
sys 0m5.069s

After enabling File Vault, the Mac restarts and it took about 50 minutes to finish the initial encryption.  While the encoding was taking place I was seeing roughly 95-120MB/sec transfer rates (190-240MB/sec combined read/write bandwidth), and it averaged about 45-50% CPU usage (% of a single core) during the encoding process.

So what how was performance with encryption enabled? Check this out…

Blade:~ mike$ time dd if=/dev/zero of=test bs=1073741824 count=10
10+0 records in
10+0 records out
10737418240 bytes transferred in 45.342448 secs (236807202 bytes/sec)

real 0m40.133s
user 0m0.001s
sys 0m4.032s

So read operations take a 5.5% hit, and write operations are 8.1% slower with File Vault 2 enabled. That’s a performance penalty I can live with for the extra peace of mind of my data being secure while traveling. Even better is the amount of CPU used was about the same whether encryption was enabled or disabled.

Performance summary:

It ended up the problem was from the LaunchServices database. The App Store checks LaunchServices to see which apps are installed. Apparently LaunchServices still had a record of an application bundle even though it had been deleted. Here’s how to use the Terminal to check and see which apps are in the LaunchServices database:

/System/Library/Frameworks/CoreServices.framework/\
Versions/A/Frameworks/LaunchServices.framework/Versions/\
A/Support/lsregister -dump

If you search through that verbose output and find an app that isn’t really there, you should rebuild the LaunchServices database. You can do that with the following command:

/System/Library/Frameworks/CoreServices.framework\
/Versions/A/Frameworks/LaunchServices.framework/Versions\
/A/Support/lsregister -kill -r -domain local -domain system -domain user

Hope this saves someone an hour or two of problem-solving…

In the file server, I configure the drives into multiple RAID 5 volumes. Right now, I have 3 RAID volumes, each with four drives. Yesterday, one of the disks in an older volume went bad. So right now I’m running 3 out of 4 drives in a RAID 5. No data loss yet, which is good. Since this is an older RAID volume, I’ve decided not to replace the failed drive. Instead, I’ll just shrink the RAID from 4 disks into 3 disks. It was quite a hassle to figure out how to do this by researching online, so I thought I would document the entire process here, step by step, to save other people some time in the future. It should go without saying that you should have a recent backup of everything on the volume you are about to change.

1. Make sure the old disk really is removed from the array. The device name shouldn’t show up in /proc/mdstat and mdadm –detail should say “removed”. If not, be sure you mdadm –fail and mdadm –remove the device from the array.

   # cat /proc/mdstat
   Personalities : [linear] [multipath] [raid0] [raid1] [raid6]...
   md0 : active raid5 sdh2[1] sdj2[0] sdi2[3]
         1452572928 blocks level 5, 64k chunk, algorithm 2 [4/3] [UU_U]
   
   unused devices: <none>
   # mdadm --detail /dev/md0
   /dev/md0:
           Version : 0.90
     Creation Time : Wed Apr  8 12:24:35 2009
        Raid Level : raid5
        Array Size : 1452572928 (1385.28 GiB 1487.43 GB)
     Used Dev Size : 484190976 (461.76 GiB 495.81 GB)
      Raid Devices : 4
     Total Devices : 3
   Preferred Minor : 0
       Persistence : Superblock is persistent
   
       Update Time : Tue Aug 16 13:33:25 2011
             State : clean, degraded
    Active Devices : 3
   Working Devices : 3
    Failed Devices : 0
     Spare Devices : 0
   
            Layout : left-symmetric
        Chunk Size : 64K
   
              UUID : 02f177d1:cb919a65:cb0d4135:3973d77d
            Events : 0.323834
   
       Number   Major   Minor   RaidDevice State
          0       8      146        0      active sync   /dev/sdj2
          1       8      114        1      active sync   /dev/sdh2
          2       0        0        2      removed
          3       8      130        3      active sync   /dev/sdi2

2. Unmount the filesystem:

   # umount /dev/md0

3. Run fsck on the filesystem:

   # e2fsck -f /dev/md0

4. Shrink the filesystem, giving yourself plenty of extra space for disk removal. Here I resized the partition to 800 GB, to give plenty of breathing room for a RAID 5 of three 500 GB drives. We’ll expand the filesystem to fill the gaps later.

   # resize2fs /dev/md0 800G

5. Now we need to actually reconfigure the array to use one less disk. To do this, we’ll first query mdadm to find out how big the new array needs to be. Then we’ll resize the array and reconfigure it for one fewer disk. First, query mdadm for a new size (replace -n3 with the number of disks in the new array):

   # mdadm --grow -n3 /dev/md0
   mdadm: this change will reduce the size of the array.
          use --grow --array-size first to truncate array.
          e.g. mdadm --grow /dev/md0 --array-size 968381952

6. This gives our new size as being 968381952. Use this to resize the array:

   # mdadm --grow /dev/md0 --array-size 968381952

7. Now that the array has been truncated, we set it to reside on one fewer disk:

   # mdadm --grow -n3 /dev/md0 --backup-file /root/mdadm.backup

8. Check to make sure the array is rebuilding. You should see something like this:

   # cat /proc/mdstat
   Personalities : [linear] [multipath] [raid0] [raid1] [raid6]...
   md0 : active raid5 sdh2[1] sdj2[0] sdi2[3]
         968381952 blocks super 0.91 level 5, 64k chunk, algorithm 2 [3/2] [UU_]
         [>....................]  reshape =  1.8% (9186496/484190976)
                                     finish=821.3min speed=9638K/sec

9. At this point, you probably want to wait until the array finishes rebuilding. However, Linux software RAID is smart enough to figure things out if you don’t want to wait. Run fsck again before expanding your filesystem back to it’s maximum size (resize2fs requires this).

   # e2fsck -f /dev/md0

10. Now do the actual expansion so the partition uses the complete raid volume (resize2fs will use the max size if a size isn’t specified):

    # resize2fs /dev/md0
    

11. (Optional) Run fsck one last time to make sure everything is still sane:

    # e2fsck -f /dev/md0

12. Finally, remount the filesystem:

    # mount /dev/md0

Everything went smoothly for me while going through this process. I could have just destroyed the entire old array and recreated a new one, but this process was easier and I didn't have to move a bunch of data around. Certainly if you are using a larger array, and are going from 10 disks to 9 or something along those lines, this benefits of using this process are even greater.

That compromise was pretty good back in 2005, but now that a half-decade has passed it is time to revisit the terrain I am including in the app. Creating a whole new terrain image set is a whole lot of work though, so I thought I would share what goes into the process here.

First, you have to find a good source of map data. For Seasonality, I’ve always liked the natural terrain look. The NASA Blue Marble imagery is beautiful, and free to use commercially, so that was an easy decision. For the original imagery I used the first generation Blue Marble imagery. Now I am using the Blue Marble Next Generation for even higher resolution.

Next you have to decide how you are going to tile the image. I’ve chosen a pretty simple tiling method, where individual tiles are 512×512 pixels, and zoom levels change by a power of 2. Square tiles are best for OpenGL rendering, and while a larger (1024, or even 2048 pixel) tile would work, 512×512 pixel tiles are faster to load into memory and if downloading over the network it will transfer faster as well. From there, you have to figure out how many tiles will be at each zoom level. I’ve chosen to use a 4×2 tile grid as a base, so the smallest image of the entire globe will be 2048 x 1024 pixels and made up of 8 tiles. As the user zooms in further, they will hit 4096×2048, 8192×4096, 16384×8192 pixel zoom levels and so on. I’ve decided to provide terrain all the way up to 65536×32768 pixels.

Now that you have an idea of what tiles need to be provided, you need to actually create the images. This is the most time consuming part of the process. Things to consider include the image format and compression amounts to use on all the tiles, and these are dependent on the type of display you are trying to generate. Creating all the tiles manually would take forever, so it’s best to automate this process.

The Blue Marble imagery comes in 8 tiles of 21600×21600 each (the full set of images for every month of the year is around 25 GB). I start by creating the biggest tile zoom level and moving down from there. For my 65536×32768 zoom level, I’ll resize each of the 8 tiles into 16384×16384 pixel images. I use a simple Automator action in Mac OS X to do this. I created an action that takes the selected files in the Finder and creates copies of the images and resizes the copies to the specified resolution.

Now that I have 8 tiles at the correct resolution, I need to create the 512×512 tiles for the final product. For Seasonality, I also need to draw all the country/state borders at this point, because otherwise the maps are blank. I created a custom Cocoa app that will read in a map image with specified latitude/longitude ranges, draw the boundaries, and write out the tiled images to a folder. My app has the restriction of only handling a single image at a time, I’ll have to drag each of the 8 tiles in separately for each zoom level. It’s not ideal, but I don’t do this too often either. For the 65536×32768 zoom level, I end up with 8192 individual tile images. Smaller zoom levels result in far fewer tiles, but you can see why automation is helpful here.

It’s a lot of work, but in the end the results are great. For Seasonality, along with higher resolution terrain, I’m also bringing in the Blue Marble’s monthly images. If everything goes as planned, Seasonality will show the “average” terrain for every month of the year. Users will be able to see the foliage change as well as the snow line move throughout the seasons.

The problem I ran into was that I was quickly hitting the maximum number of files on the filesystem.  The database I’m working on creates millions of files to store its data in, and I was quickly running out of inodes.

Earlier today I installed a fresh copy of Ubuntu on a virtual machine where the final forecast server will reside.  Of course I forgot to increase the number of inodes before installing the OS on the new partition.  Unfortunately, there is no way to add more inodes to a Linux ext4 filesystem without reformatting the volume.  Luckily I caught the problem pretty early and didn’t get too far into the system setup.

To fix the issue, I booted off the Ubuntu install ISO again and chose the repair boot option.  Then I had it start a console without selecting a root partition (if you select a root partition, it will mount the partition and when I tried to unmount it, the partition was in use).  This let me format the partition with an increased number of inodes using the -N flag in mkfs:

mkfs.ext4 -N 100000000 /dev/sda1

That ought to be enough.  After that, I was able to install Ubuntu on the new partition (just making sure not to select to format that same partition again, wiping out your super-inode format).

The forecast server is coming along quite well.  I’m hoping to post more about how it all works in the near future.

Last year I bought a new server to replace the 5 year old weather server I was using at the time. This server is being coloed at a local ISPs datacenter. I ended up with a Dell R710 with a Xeon E5630 quad-core CPU and 12GB of RAM. I have 2 mirrored RAID volumes on the server. The fast storage is handled by 2 300GB 15000 RPM drives. I also have a slower mirrored RAID using 2 500GB 7200 RPM SAS drives that’s used mostly to store archived weather data. The whole system is running VMware ESXi with 5-6 virtual machines, and has been working great so far.

Adding this new server meant that it was time to bring the old one back to the office. For its time, the old server was a good box, but I was starting to experience reliability issues with it in a production environment (which is why I replaced it to begin with). The thing is, the hardware is still pretty decent (dual core Athlon, 4GB of RAM, 4x 750GB disks), so I decided I would use it as a development server. I mounted it in the office rack and started using it almost immediately.

A development box really doesn’t need a 4 disk RAID though. I currently have a Linux file server in a chassis with 20 drive bays. I can always use more space on the file server, so it made sense to consolidate the storage there. I moved the 4 750GB disks over to the file server (setup as a RAID 5) and installed just a single disk in the development box. This brings the total redundant file server storage up past 4 TB.

The next change was with the network infrastructure itself. I have 2 Netgear 8 port gigabit switches to shuffle traffic around the local network. Well, one of them died a few days ago so I had to replace it. I considered just buying another 8 port switch to replace the dead one, but with a constant struggle to find open ports and the desire to tidy my network a bit, I decided to replace both switches with a single 24 port Netgear Smart Switch. The new switch, which is still on its way, will let me setup VLANs to make my network management easier. The new switch also allows for port trunking, which I am anxious to try. Both my Mac Pro and the Linux file server have dual gigabit ethernet ports. It would be great to trunk the two ports on each box for 2 gigabits of bandwidth between those two hosts.

The last recent network change was the addition of a new wireless access point. I’ve been using a Linksys 802.11g wireless router for the last several years. In recent months, it has started to drop wireless connections randomly every couple of hours. This got to be pretty irritating on devices like laptops and the iPad where a wired network option really wasn’t available. I finally decided to break down and buy a new wireless router. There are a lot of choices in this market, but I decided to take the easy route and just get an Apple Airport Extreme. I was tempted to try an ASUS model with DD-WRT or Tomato Firmware, but in the end I decided I just didn’t have the time to mess with it. So far, I’ve been pretty happy with the Airport Extreme’s 802.11n performance over the slower 802.11g.

Looking forward to finalizing the changes above. I’ll post some photos of the rack once it’s completed.

First, open an ASDM connection to your router. Go into the Configuration screens and click on Firewall to configure the firewall options. Then click on Service Policy Rules to configure the services that the firewall software will monitor. Select the global policy (first and only one in the list), and click on the Edit button. Switch to the Rule Actions (3rd) tab, and in the list check to enable ICMP. You can leave ICMP Error unchecked. Close that and Apply the changes.

Now, if you just want to be able to ping, stop here and you are done. However, traceroute will not work with this setup. For traceroute to work, you have to complete this follow-up task.

While still under the Firewall configuration switch to the Access Rules item. Add an access rule to permit ICMP traffic. Click the Add button, make sure the interface is set to outside, action is Permit, and Source/Destination is any. Under Service, click the … button and select the icmp line and click OK. Click OK again in the Add Access Rule dialog and Apply the results to finish the process.