The new forecast server I’m working on for Seasonality users is using the filesystem heirarchy as a form of database instead of PostgreSQL. \u00a0This will slow down the forecast generation code a bit, because I’m writing a ton of small files instead of letting Postgres optimize disk I\/O. \u00a0However, reading from the database will be lightning fast, because filesystems are very efficient at traversing directory structures.<\/p>\n
The problem I ran into was that I was quickly hitting the maximum number of files on the filesystem. \u00a0The database I’m working on creates millions of files to store its data in, and I was quickly running out of inodes.<\/p>\n
Earlier today I installed a fresh copy of Ubuntu on a virtual machine where the final forecast server will reside. \u00a0Of course I forgot to increase the number of inodes before installing the OS on the new partition. \u00a0Unfortunately, there is no way to add more inodes to a Linux ext4 filesystem without reformatting the volume. \u00a0Luckily I caught the problem pretty early and didn’t get too far into the system setup.<\/p>\n
To fix the issue, I booted off the Ubuntu install ISO again and chose the repair boot option. \u00a0Then I had it start a console without selecting a root partition (if you select a root partition, it will mount the partition and when I tried to unmount it, the partition was in use). \u00a0This let me format the partition with an increased number of inodes using the -N flag in mkfs:<\/p>\n
mkfs.ext4 -N 100000000 \/dev\/sda1<\/pre>\nThat ought to be enough. \ud83d\ude42 \u00a0After that, I was able to install Ubuntu on the new partition (just making sure not to select to format that same partition again, wiping out your super-inode format).<\/p>\n
The forecast server is coming along quite well. \u00a0I’m hoping to post more about how it all works in the near future.<\/p>\n","protected":false},"excerpt":{"rendered":"
The new forecast server I’m working on for Seasonality users is using the filesystem heirarchy as a form of database instead of PostgreSQL. \u00a0This will slow down the forecast generation code a bit, because I’m writing a ton of small files instead of letting Postgres optimize disk I\/O. \u00a0However, reading from the database will be […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[27,28,29,30],"tags":[],"class_list":["post-264","post","type-post","status-publish","format-standard","hentry","category-seasonality","category-seasonality-go","category-storage","category-sys-admin","post-preview"],"_links":{"self":[{"href":"https:\/\/www.starcoder.com\/wordpress\/wp-json\/wp\/v2\/posts\/264","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.starcoder.com\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.starcoder.com\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.starcoder.com\/wordpress\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.starcoder.com\/wordpress\/wp-json\/wp\/v2\/comments?post=264"}],"version-history":[{"count":4,"href":"https:\/\/www.starcoder.com\/wordpress\/wp-json\/wp\/v2\/posts\/264\/revisions"}],"predecessor-version":[{"id":568,"href":"https:\/\/www.starcoder.com\/wordpress\/wp-json\/wp\/v2\/posts\/264\/revisions\/568"}],"wp:attachment":[{"href":"https:\/\/www.starcoder.com\/wordpress\/wp-json\/wp\/v2\/media?parent=264"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.starcoder.com\/wordpress\/wp-json\/wp\/v2\/categories?post=264"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.starcoder.com\/wordpress\/wp-json\/wp\/v2\/tags?post=264"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}