Roderick Derks

The commands vicfg-cfgbackup.pl (esxcfg-cfgbackup.pl) allow you to backup and restore the configuration of your ESX 3i host. Install the RCLI on your pc, download it by clicking the link on the webpage on an ESXi host.

To backup the host you would run the command.

vicfg-cfgbackup.pl –server <server_name> -s <backup_file_name>

To restore your backup configuration to your host you would run the following command. This will cause the host to reboot once the process is complete.

vicfg-cfgbackup.pl –server <server_name> -l <backup_file_name>

NOTES

– While this command can be used to restore your configuration, it is not a substitute for backup of your virtual machines. At this time, a new install of ESX 3i Installable will wipe out any existing datastores that exist on the host when you perform a new install.

– You can use the -q switch to supress the confirmation request that you will receive when you restore a configuratio backup or restore a host back to factory defaults.

– You must have all VMs on the host stopped and have put the host in maintenance mode. Also the build of the install on the ESX host must match the build that is recorded in the backup file. If that is not the case, you can use the -f switch to force the restore. This document is based on the 0.26 version of the RCLI and at this point both the maintenance mode and build number requirements are not enforced.

vicfg-cfgbackup.pl –server <server_name> -f -l <backup_file_name>

To reset the host backup to factory defaults, use the below command. It will put the host in maintenance mode and it will not wipe out any existing datastores. Thus you will be able to add an existing virtual machines back into inventory by either manually adding the virtual machines or by restoring a configuration backup.

vicfg-cfgbackup.pl –server <server_name> -r

More online info about the RCLI commands you can find here

I just had to create a couple VMFS volumes from the command-line on a local disk. It’s pretty straight forward but I needed to document it anyway, here it is:

1. fdisk -l (check what the last partition is)
2. fdisk /dev/disks/vmhba1\:0\:0\:0
3. d (I had to delete formerly created linux partitions)
4. n (new)
5. default (start block)
6. default (end block)
7. t (partition type)
8. fb (vmfs type)
9. 1. X
   2. b
   3. 1
   4. 128 (disk alignment, check your SAN manual for the correct value, 128 is correct in most cases…)
   5. W
10. reboot if needed (I did not have to reboot on my ESXi USB VMware server)
11. ~ # vmkfstools -C vmfs3 /dev/disks/vmhba1\:0\:0\:1
    Creating vmfs3 file system on “vmhba1:0:0:1” with blockSize 1048576 and volume label “none”.
    Successfully created new volume: 49ce3608-2b378b66-c502-00123fba3df2
12. Go to VI Client, Refresh storage view and rename the newly created volume

This artice describes my Freenas experiences since march 2009.

From VMware Server to ESXi and a NAS solution (March 2009)

December 2008 after using VMware Server for 3 years I decided that it was time to switch to ESXi. That was a good moment to start experimenting with a NAS. And offcourse because my network is very low budget I was looking for a free open source alternative that could run on simple hardware.By the way, in these three years of using VMware Server to host my website and emailserver at home, I just had two reboots for maintenance reasons. Uptime of the virtual and physical servers was over 300 days 🙂

My setup:
– Two ESX servers: Pentium4 2GB RAM and each one has a 1GB NIC.
– Server used to install FreeNas and OpenFiler: Pentium4, 1GB RAM, 1GB NIC
– Switch 1GB
 

FreeNas

I started with installing FreeNas as I played with this software in the past. When installing I was pleasently surprised by the options that FreeNas offered me. You can boot it from the cd, install it on the harddrive and boot it from there, or install it on an USB drive and then boot it. FreeNas installer also gave me options to upgrade an older installed version on the harddisk or USB device, very nice.

Installing was very simple, booting FreeNas is very fast and configuring is very easy too if you play with it for a while. The webgui is fast and looks professional.

I created an iSCSI LUN and presented it to the ESX's. That worked pretty good. First connected it to ESX1 and then to ESX2. I created a VMFS file system via the ESX2 and then created virtual machines.

Then I ran into a problem. I noticed that the LUN was not available anymore on my ESX1 server, just on my ESX2. I tried to rescan, but I still could not add it anymore. Then I noticed that the ESX1 storage adapter could see the LUN, but I just could not add it anymore as a storage. After some google-time the problem seemed to be that FreeNas can not handle Shared Storage. And that means No VMotion! That was a bummer.

Another issue I ran into was the performance. I have the idea FreeNas is not superfast in my setup at home. The VM's are responding kind of slowish. I also used SMB/CIFS to share files from another disk and this was not fast too compared to my Fedora SAMBA setup that I had been using for  couple of years uptill then.
 

OpenFiler

I decided I had to experiment with another NAS and I wanted to try OpenFiler. Openfiler is a well known open source project and it has shared storage feature. The problem was where to find the time in my life to do this. So yesterday late in the evening I decided to start, and I finished early in the morning.

The installation is easy, not as luxureous as the FreeNas install but good. My monitor could not handle the graphic install so I had to do it via the text gui. No problem. The nice upgrade features that FreeNas has are not available in OpenFiler, and installing on a USB Memory Stick device is not supported too. The OS is bigger then the FreeNas installation. I used a 2GB partition on the harddisk for the system partition and the rest of the space I left free to create a local VMFS. OpenFiler installation uses about 600MB on the disk. After installation I connected my webbrowser using https on port 446 to the webgui, which is not as fast and as cool as FreeNas, but it works fine. Default account is openfiler with password password.

I had my USB 1TB disk that contained a partition with a VMFS filesystem and some VM's. I connected it to OpenFiler, the disk was found but I could not use it. The problem is that OpenFiler expects a disk to be partitioned with Logical Volume Manager. If not then you can not connect to existing filesystems on the disk. This was pretty uncool cause in FreeNas it is really easy to add a disk with an existing partition and use the data that is stored on it. Doesn't matter if it is ETX2, ETX3, NTFS filesystem or that you present it raw to an ESX server when you have a VMFS volume on it.

So I connected the USB 1TB VMFS disk again to a FreeNas server (that I installed on a really old PIII pc), created an iSCISI target and connected it to an ESX. I copied the VM's from it and then deleted the VMFS partition.

Then I connected the 1TB disk to OpenFiler again, used LVM to create a partition and then created an iSCSI LUN. Connected both ESX servers to it..  et voilà: shared storage!

I got OpenFiler running now for one day and I can see that it works fine. And VMotion works! Now I'm able to meet the SLA agreement that I made with myself 😉 It might be interesting to experiment with NFS too to see the performance difference with iSCSI.

There is just one other downside not related to VMware when you compare FreeNas to OpenFiler: with FreeNas I used another 220GB USB drive to share my files that are on a EXT3 filesystem via SMB/CIFS. Another 215GB USB disk I used as a backup for this disk, synchronizing it using RSYNC in FreeNas. That really works great!
As these two disks that I mentioned are not partitioned using Logical Volume Manager, OpenFiler can not find the existing EXT3 filesystems. I have to format one of the disks, use LVM to create a partition, format it to EXT3 and then copy the data. And then do the same for the second disk.
After doing this I have to find out if I can sync the two disks using RSYNC inside OpenFiler, and I'm not sure that this is possible. And then see if the performance is better then Freenas when I use SMB/CIFS to share my files. I'll let you know how this works out when I find the time to try it out.

Conclusions for now

If you want your VMware ESX servers to have shared storage so that you can VMotion then you have to use OpenFiler and not Freenas. OpenFiler works great and is not hard to setup.
FreeNas is more userfriendly and there is more eye for detail in the webgui. Future upgrade situations are handled in the installation phase. FreeNas makes it easy to add disks with existing partitions and filesystems on it when LVM is not used. OpenFiler prefers an empty disk and wants to create partitons using LVM. You can not add a disk with existing LVM's and existing partitions without doing some tricks (which I haven't tried).

Let me know if you find this article usefull, makes me happy 🙂 HaveANiceDay

More then one year later… (august 2010)

Ok, I wrote this article in March 2009. Now it's august 2010 and I have the following situation at home:

• One ESX 4.1 server (instead of two) – new hardware that costed me about €400,- with a Xeon CPU with 4 cores, 8GB Mem and it performs very well and consumes less power
• Six VM's running, mostly linux (no X windows) and also a W2008 (x64) and I still got resources left to experiment
• One of the six VM's is Freenas. I attached a 500GB VMDK to it and this storage contains all of my files (documents, music, etc) And it's my bittorrent server too.
• I used one of the old ESXi servers (desktop pc) on which I installed Freenas, and attached an external USB disk to it. So this is my backup server for my data and I use RSYNC to copy files between the two Freenas instances during the night now and then. And I use VMware's Data Recovery to backup my VM's to the Freenas Backup server.

So as you can see I started to use Freenas more then Openfiler. Why is that? Well, it's it's just a little more userfriendly and easier to set up. You can do everything in the GUI and since I have to be efficient with my time I used freenas. On my webserver I created a "proxy website" so I can connect to and configure my freenas servers from the internet when I'm not at home.
A backup of the Freenas config can be made so in case of a problem a restore is no sweat. Freenas can handle Shared Storage LUN's nowadays so that is cool too. 

Almost two years later… (januari 2011)

Recently for the first time in my life I bought a new TV, and this one is able to play files from a media server. And yes, Freenas has a builtin media server: Fuppes. I configured it and it works! If you don't have a TV with a direct media server connection you can always use a PS3, 360 Xbox and more.

The Media Server feature combined with the torrent feature in Freenas is great. First download a movie and them play it on your TV using the media server.

I only have a problem: subtitles. The .srt files are not recognized by my TV and therefor not used. For a Dutch guy a movie like The Social Network is hard work to understand everything.

By the way: both my Virtual and Physical Freenas system have not crashed yet, nor has VMware ESX 4.1 and all of my other Linux servers running on top of ESX (251 days uptime).

Well, hope you can use this info. Leave a message if you'd like. I'd appreciate it to hear your idea's.

ESXi5 and Freenas iSCSI (September 2011)

I read about the incompatibility of Freenas and ESXi5 iSCSI. On September 2nd the latest Freenas version wat released with fixes for iSCSI.

• Upgraded Freenas 0.7.1 version to 0.7.2.7529 Sabanda. The upgrade failed, but I was warned in the readme file for this.
• Installed Freenas from scratch and restored my old backupped configuration. This worked. So after 15 minutes I could try to connect an iSCSI target to a ESXi5 host. I just needed an ESXi5 host.
• Installed ESXi5 on my ESXi4 platform in a guest and connected it to the iSCSI target. This seemed to work.
• Created a guest with the VMDK on the iSCSI target and linux as a OS, this worked too.

Another job well done by Freenas.

Problem

Release 5.0.14_GA_2850.F7_20090303145157 F7

I can not contact the Zimbra server anymore using a Zimbra client or the website.

# tail -f /var/log/zimbra.log
Mar 28 01:12:15 my_servername zimbramon[2461]: 2461:info: zmmtaconfig: Sleeping…Key lookup failed.
Mar 28 01:12:16 my_servername postfix/smtpd[6505]: warning: problem talking to service rewrite: Success
Mar 28 01:12:16 my_servername postfix/master[3629]: warning: process /opt/zimbra/postfix/libexec/trivial-rewrite pid 7850 exit status 1
Mar 28 01:12:16 my_servername postfix/master[3629]: warning: /opt/zimbra/postfix/libexec/trivial-rewrite: bad command startup — throttling
Mar 28 01:12:22 my_servername zimbramon[2461]: 2461:info: zmmtaconfig: Skipping Global system configuration update.
Mar 28 01:12:22 my_servername zimbramon[2461]: 2461:info: zmmtaconfig: gacf ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: java.net.SocketException Broken pipe)
Mar 28 01:12:28 my_servername zimbramon[2461]: 2461:info: zmmtaconfig: Skipping All Reverse Proxy URLs update.
Mar 28 01:12:28 my_servername zimbramon[2461]: 2461:info: zmmtaconfig: Skipping getAllReverseProxyURLs ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: java.net.SocketException Broken pipe)
Mar 28 01:12:32 my_servername zimbramon[2461]: 2461:info: zmmtaconfig: Skipping All Reverse Proxy Backends update.
Mar 28 01:12:32 my_servername zimbramon[2461]: 2461:info: zmmtaconfig: Skipping getAllReverseProxyBackends ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: java.net.SocketException Broken pipe)
Mar 28 01:12:35 my_servername zimbramon[2461]: 2461:info: zmmtaconfig: Skipping All Memcached Servers update.
Mar 28 01:12:35 my_servername zimbramon[2461]: 2461:info: zmmtaconfig: Skipping getAllMemcachedServers ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: java.net.SocketException Broken pipe)
Mar 28 01:12:38 my_servername zimbramon[2461]: 2461:info: zmmtaconfig: Skipping All MTA Authentication Target URLs update.
Mar 28 01:12:38 my_servername zimbramon[2461]: 2461:info: zmmtaconfig: Skipping getAllMtaAuthURLs ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: java.net.SocketException Broken pipe)
Mar 28 01:12:42 my_servername zimbramon[2461]: 2461:info: zmmtaconfig: Skipping Configuration for server my_servername.r71.nl update.
Mar 28 01:12:42 my_servername zimbramon[2461]: 2461:info: zmmtaconfig: gs:my_servername.r71.nl ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: java.net.SocketException Broken pipe)
Mar 28 01:12:42 my_servername zimbramon[2461]: 2461:info: zmmtaconfig: Sleeping…Key lookup failed.

Hostfiles seem to be OK, DNS works great… ip address did not change…
I tried to follow up advice in the ZIMBRA forum but I could not find a solution.

Problem started after the time I restarted ZIMBRA for the first time after the upgrade, so first it worked after the upgrade!

Solution

Well… this one took me a while.

I had to recreate the certificate. The guide was very helpfull.

Then I ran into the problem the MTA would not start anymore:

[zimbra@my_servername ~]$ zmcontrol start
Host ventoux.r71.nl
Starting ldap…Done.
Starting logger…Done.
Starting mailbox…Done.
Starting antispam…Done.
Starting antivirus…Done.
Starting snmp…Done.
Starting spell…Done.
Starting mta…FAILED
Starting zmmtaconfig…zmmtaconfig is already running.
postalias: warning: /etc/aliases, line 97: need name:value pair
postsuper: fatal: scan_dir_push: open directory defer: Permission denied
postfix failed to start
Starting saslauthd…done.
Starting stats…Done.

Solution to this problem:
# /opt/zimbra/libexec/zmfixperms (run as root)

I’m back in business again.

Change the following lines in /etc/mail/sendmail.mc

define(`SMART_HOST’, `ventoux.r71.nl’)dnl

dnl DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA’)dnl
DAEMON_OPTIONS(`Port=smtp, Name=MTA’)dnl

LOCAL_DOMAIN(`alpedhuez.r71.nl’)dnl

  # m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf

Flush deferred messages in sendmail queue

Whenever sendmail has to deliver mails to other hosts which cannot be reached at that time, the messages are kept in the queue and are marked as “Deferred: Connection timed out”. Although the other hosts could be reached again and you want to tell sendmail to flush the mail queue, the command

  # sendmail -q -v

does not really try to reconnect to these hosts and still assumes that the connection timed out. The reason is that the hoststatus is cached, per default for a period of 30 minutes. Using

  # sendmail -OTimeout.hoststatus=0m -q -v

You can re-run the mail queue and force sendmail to reconnect to the hosts. You may want to define an alias for that, say, ‘sendmail-flush-timeouts’.

Default mail queue interval setting

  #  cat /etc/sysconfig/sendmail
      DAEMON=yes
      QUEUE=1h

Test on commandline

telnet yourrelayserver.com 25

helo <hostname>
mail from: name@extrnaldomain.com
rcpt to: user@yourdomain.com
data
mail test
.