# Category: Nerd stuff

Just stumbled upon this by accident, and it’s no massive thing… but a ^Z will minimise the gvim window, at least in the MS Windows version.

It allows people that shouldn’t be coding, to write horrid code.

Given the long history I have with servers and SAN storage, the only time I’ve ever configured boot from SAN is for Solaris SPARC boxes.

We recently bought new B200 M3 blades for our UCS infrastructure, and with that order I unintentionally omitted ordering local disks with them (we usually do a basic RAID 0 across two local disks to boot ESXi from).

I was actualy ok with this, as we have an amazing EMC SAN, and having less disks to die, and less power to consume, is always a good thing.  So, this was the first time I’d implemented booting ESXi from SAN.

The issue was that vSphere was telling me “System logs are stored on nonpersistent storage”, which is fair if you care about system logs surviving a host reboot, which I do.

# Easy setup

Though beyond the scope of this post, it’s worthy to note that actually doing it is quite simple — create a new LUN, zone it in the fabric appropriately, make sure it’s LUN0 and off you go.

# Size matters

I discovered after doing one or two that the sizing of the boot LUN matters, from the perspective of a concept in ESXi called persistent storage.  In short, if ESXi doesn’t have enough space for its scratch partition on “persistent” storage (ie. a local disk, a SAN disk, etc), it’ll create one in the host RAMdisk on each boot (therefore being volatile).  This partition size is, of course, determined when you install ESXi, and isn’t easily changed after install — it’s actually quicker to just reinstall.

Installing ESXi 5.1 requires a boot device that is a minimum of 1GB in size. When booting from a local disk or SAN/iSCSI LUN, a 5.2GB disk is required to allow for the creation of the VMFS volume and a 4GB scratch partition on the boot device. If a smaller disk or LUN is used, the installer attempts to allocate a scratch region on a separate local disk. If a local disk cannot be found, the scratch partition (/scratch) is located on the ESXi host ramdisk, linked to /tmp/scratch. You can reconfigure /scratch to use a separate disk or LUN. For best performance and memory optimization, VMware recommeds that you do not leave /scratch on the ESXi host ramdisk.

My mistake was to just pick a rounded off size, eg. 5GB, for my boot LUNs.  I was 200MB off 😉

# The magic number

Sizing the LUN just a little bit bigger to satisfy (or perhaps trick) ESXi into doing a fully-local install is the trick.  I’m a purist, and refused to just goto, say, 6GB, so I experiented a little and pretty much fluked it first go.

The challenge for me was how to calculate 5.2GB into MB, because EMC Unisphere doesn’t accept decimals.  Then there’s the issue of “does this ‘thing’ calculate units in powers of 2 or not”?  Turns out the math was simple:

$5.2 \times 1024 {=} 5{,}324.8$

Round that up to 5325MB and ESXi likes that, and partitions the disk thusly:

vfat         4.0G  31.8M      4.0G   1% /vmfs/volumes/<long ID>
vfat       249.7M 130.2M    119.5M  52% /vmfs/volumes/<long ID>
vfat       249.7M   8.0K    249.7M   0% /vmfs/volumes/<long ID>
vfat       285.8M 201.9M     83.9M  71% /vmfs/volumes/<long ID>


Which loosely translates into the following mount point for /scratch (sadly, there is no mount command in ESXi):

lrwxrwxrwx 1 root root 49 Apr 2 08:15 scratch -> /vmfs/volumes/<long ID>



If the storage was on non-persistent media, that would be linked to /tmp.

# The other method

Some like to actually use a shared datastore (eg via NFS), using a seperate subdirectory for each host, and changing the Syslog.global.logDir variable to point to it.  The main goal there is to save SAN LUN space.  Personally, I prefer the logs to be with the host, as it were, on it’s own LUN.

I was recently faced with the task that a lot of experienced network admins have to deal with — being able to execute multiple commands on a router/switch, but commands that may temporarily break connectivity to said device as each line is pasted.

# In the past…

Traditionally, one would achieve this “device-side chained execution” by using TFTP, as the “script” is only executed once uploaded to the device.  Conceptually, this is the same as being able to chain multiple commands at a Unix shell prompt, delimiting them with a semicolon.

# Why oh why, silly auto-negotiate?!?

My example of needing to do this lately was to remotely change an interface’s speed and duplex — commands that typically knock ones telnet/ssh out for a period of time, or completely.  They were both set to auto, but was ethernet on one side, and gigabit ethernet on the other… whicih is always a major fail in autonegotiation land.

Of course, the seasoned router guy, purely by habit, will schedule a “reload in” just in case, but in their rare wisdom, Cisco brought the wonderfully-underrated language Tcl to their codebase.

# Well hello, Tcl!

I’ve got a long and wonderous history with Tcl, and it’s graphical partner, Tk.  I forgot Tcl was available on more recent IOSs.  After some googling, I found though that a lot of sites didn’t talk about using it for remote work, where you need the classic chaining of commands.

Enough jabbering on.  This is how I used it for the speed and duplex change:

router#tclsh
router(tcl)#ios_config "interface gi0/1" "speed 100" "duplex full"
router(tcl)#


You may even want to combine that with a “reload in” 😉

So I’m a huge fan of fwbuilder, having been a security guy for many years, and having been spoilt by Checkpoint’s SmartDashboard GUI (one of the few things they do well).

Up until recently, I’ve not really needed a GUI for firewall management as we did most firewalling via the CP boxes.  Now, with the advent of having to admin over 150 (and growing) Linux VMs, and the iptables instances therein, fwbuilder fits the bill perfectly.

# Great design

One of the best things about fwbuilder is its use of XML for configuration files.  This means all sorts of useful fun can be had when bulk changes are needed.  Still, XML can be hard at the best of times, due to the multi-line structure of it — a topic for another day of hackery.

# Bulk import/discovery

fwbuilder has a few options in this regard, and I ended up using SNMP, so that the interfaces themselves (and names) are also brought in.  The other option was a painful import of the iptables save/restore stuff, but that would have still meant manual, or automated hackery, to config interfaces.

The issue is that the imports are left pretty raw; you can’t choose a firewall type, which is the management interface, and so on.  fwbuilder is lacking in this area of bulk change — fwbedit gets you some of the way, but not all of the way.

# Good ole sed

I’m a Unix guy; always have been, always will be.  One of my favourite things is regexps, and with that, sed.

Here’s what I used to at least bulk change platform and host_OS on the XML:

sed -ie '/platform="unknown" name="au-/s@platform="unknown"@platform="iptables"@; s@host_OS="unknown"@host_OS="linux24"@' yourconfig.fwb


It’s a little lazy, but does the job, using the unknown text as an anchor.  The part of the pattern matching for au- is just a common starting name for all of our firewalls.

# fwbedit

The next issue was bulk-changing the management interface of all the 150+ firewalls.  fwbedit has this capability, but the documentation isn’t the best, hence this blog post.

This format seems to have worked for me:

fwbedit.exe modify -f yourconfig.fwb -o /User/Firewalls/somefirewall -a 0,,1


The syntax for doing modify’s is not all that clear.  The doco says:

modify -f file.fwb -o object -c comment [-a attrs]

Modifies object specified by its full path in the tree or object ID. Object can not be renamed using this operation.
-f file.fwb: data file
-o object: object to be deleted, full path or ID
-c txt:  specify comment for the new object
-a attribute1[,attribute2…]  :  specify attributes that
define parameters of the new object (see below)

A few things here — firstly, the comment argument appears entirely optional, but isn’t indicated thusly via the traditional use of square brackets around it.  Secondly, the attribute stuff is confusing:

-t Interface -a security level,address type (dynamic or unnumbered),management

It doesn’t show here what arguments are mandatory, their format (boolean, “true/false” etc).  After hunting the source, using integers is what seems to work.

Now, with that in mind, we loop over all management interfaces, and set them right:


fwbedit.exe list -f yourconfig.fwb -r -o /FWObjectDatabase/User/Firewalls | grep '.*au-.*/eth0$' | while read id; do fwbedit.exe modify -f yourconfig.fwb -o$id -a 0,,1; done



Here we’re just iterating over the firewalls we’re interested in to get their ID, and then modify accordingly.

I hope this helps someone 🙂

Page 2 of 2

Powered by WordPress & Theme by Anders Norén