I host my web content on a samba share. When shutting down the apache server, it takes a very long time due to the fact that it stops SAMBA before stopping apache 
Therefore stopping apache takes almost 15 minutes while the lookups time out.
grrrrr.
Earlier today I was going to copy some virtual machines onto a USB drive for the upcoming Defcon conference. I ran dmesg to see what the device name of the drive was, and my logs were filled with lost interrupt messages:
[ 8546.803754] rtc: lost some interrupts at 512Hz.
[ 8546.823739] rtc: lost some interrupts at 512Hz.
[ 8546.843730] rtc: lost some interrupts at 512Hz.
I did some research and came across:
this post and it linked to this vmware documentation.
I have done the following things to improve performance:
1. Set tools.timeSync = true in /etc/vmware/config.
2. Turned off NTP and w32time in the guests and configured NTP on the host.
3. Switched to the server kernel on the host and guest kernels. I am running Ubuntu Feisty (7.04) on the host and it was using the generic kernel for some reason. My kernel version is: Linux vmware-server 2.6.20-15-server #2 SMP Sun Apr 15 06:22:36 UTC 2007 x86_64 GNU/Linux
4. Set /proc/sys/vm/swappiness to 0. I have 4 gigabytes of ram on this server so there shouldn’t be any need to swap ever! 
5. Set vmware host memory settings (from the vmware console under the host->settings->memory tab) to allow some virtual memory to be swapped (the second radio button option). I set host ram for virtual machines as high as it could go. (Screenshot)
6. Set host.useFastClock = FALSE and monitor_control.virtual_rdtsc = FALSE in /etc/vmware/config. This eliminates the RTC messages on the host, and greatly reduces the number of context switches.
7. Installed vmware tools in all guests.
8. Set all of my guests to use only 1 CPU.
For more information on linux performance tuning please see these excellent texts. I am still learning about the topic myself.
So after making all of these changes I restarted vmware. Now for performance statistics. I gathered some rough ones before making the changes under a fairly typical workload (a few linux and windows guests running doing moderate levels of activity). The load was hovering around 5 and the context switches were 20k+.
While running all of thebelow tests I am running another instance of vmstat sampling every second to catch any major spikes that happen between sampling intervals.
Here are the stats after all machines have been booted and all services verified as running:
First the running virtual machines:
charles@vmware-server:~/vmware-machines/File Server$ ps -ef | grep vmx
charles 10979 1 6 20:02 ? 00:00:38 /usr/lib/vmware/bin/vmware-vmx -C /vmware-prod/home/charles/vmware-machines/Internal Server/Internal Server.vmx -@ “”
charles 10999 1 3 20:02 ? 00:00:18 /usr/lib/vmware/bin/vmware-vmx -C /vmware-prod/home/charles/vmware-machines/VPN Server/VPN Server.vmx -@ “”
charles 11022 1 3 20:02 ? 00:00:19 /usr/lib/vmware/bin/vmware-vmx -C /vmware-prod/home/charles/vmware-machines/PrepRep/PrepRep.vmx -@ “”
charles 11079 1 5 20:04 ? 00:00:22 /usr/lib/vmware/bin/vmware-vmx -C /vmware-prod/home/charles/vmware-machines/File Server/File Server.vmx -@ “”
charles 11103 1 17 20:05 ? 00:01:00 /usr/lib/vmware/bin/vmware-vmx -C /vmware-prod/home/charles/vmware-machines/W2k3 App Server/W2k3 App Server.vmx -@ “”
charles 11129 1 11 20:07 ? 00:00:26 /usr/lib/vmware/bin/vmware-vmx -C /vmware-prod/home/charles/vmware-machines/Public Server/Public Server.vmx -@ “”
charles 11153 1 16 20:09 ? 00:00:22 /usr/lib/vmware/bin/vmware-vmx -C /vmware-prod/home/charles/vmware-machines/W2K3-Production/W2K3-Production.vmx -@ “”
charles 11184 10451 0 20:11 pts/0 00:00:00 grep vmx
charles@vmware-server:~/vmware-machines/File Server$
And now the performance:
charles@vmware-server:~$ vmstat 10 30
procs ———–memory———- —swap– —–io—- -system– —-cpu—-
r b swpd free buff cache si so bi bo in cs us sy id wa
1 0 140 26500 37580 3338504 0 0 92 197 75 290 4 24 71 1
5 0 140 26120 37620 3338956 0 0 28 489 195 3400 0 4 93 3
6 0 140 25004 37644 3338980 0 0 1 826 166 2902 0 1 99 0
12 0 140 23952 37664 3339052 0 0 0 31 118 2839 0 0 99 0
5 0 140 23828 37672 3339052 0 0 0 18 116 2828 0 0 99 0
1 0 140 25832 37652 3335664 0 0 2 658 145 2907 0 1 99 0
5 0 140 26080 37676 3335668 0 0 1 72 122 2843 0 1 99 0
7 0 140 25708 37684 3335696 0 0 2 21 119 2864 0 1 99 1
3 0 140 25336 37712 3335712 0 0 2 52 121 2848 0 1 98 1
3 0 140 23972 37740 3335748 0 0 2 800 162 2921 0 1 97 2
6 0 140 24104 37756 3336148 0 0 1 72 120 2877 0 0 98 1
6 0 140 23856 37768 3336180 0 0 2 52 119 2875 0 1 99 0
8 0 140 25740 37368 3333484 0 0 1 1128 160 2937 0 1 98 1
5 0 140 25864 37388 3333660 0 0 14 706 189 3405 0 2 95 3
5 0 140 25244 37412 3333680 0 0 2 54 133 2898 0 1 99 0
5 0 140 24748 37432 3333724 0 0 1 81 124 2867 0 1 99 0
6 0 140 27376 37456 3333744 0 0 1 1086 162 2889 0 1 98 1
3 0 140 27000 37480 3335288 0 0 6 148 174 3031 0 5 94 2
3 0 140 24776 37512 3337892 0 0 44 152 234 3086 0 3 96 1
2 0 140 28008 37436 3334380 0 0 166 1179 192 3013 0 2 96 1
3 0 140 27264 37476 3334432 0 0 7 80 141 2903 0 1 99 1
5 0 140 26272 37500 3334472 0 0 2 498 119 2899 0 2 98 0
4 0 140 25776 37516 3334488 0 0 2 641 136 2840 0 1 99 0
6 0 140 25532 37536 3334636 0 0 13 448 140 3779 1 2 95 2
6 0 140 25036 37560 3334656 0 0 1 670 129 2848 0 1 99 0
6 0 140 25160 37580 3334684 0 0 2 198 130 2879 0 0 99 0
5 0 140 24416 37600 3334712 0 0 1 824 149 2877 0 1 99 0
6 0 140 23548 37632 3334764 0 0 5 410 156 2957 0 2 98 0
5 0 140 23548 37640 3334796 0 0 1 41 109 2863 0 1 99 0
1 1 140 24816 37552 3333048 0 0 27 1106 181 3069 0 3 94 3
charles@vmware-server:~$
I highly recommend doing the above items on Vmware Server and possibly on ESX server as well. Performance of my server farm is vastly improved.
So a while ago, I worked on some scripts to assist with management of VmWare server guests. I have been looking into how the other virtulization solutions handle management. There appears to be a strong preference toward CIM and DMTF. Having done a lot of work with WMI (the Microsoft implementation of DMTF and WBEM), I am really happy that they all support it.
Some more information:
Microsoft (Virtual Server and Viridian): http://download.microsoft.com/download/5/b/9/5b97017b-e28a-4bae-ba48-174cf47d23cd/VIR043_WH06.ppt#305,21,ManagementService
Xen:
http://wiki.xensource.com/xenwiki/XenCim
Vmware (ESX anyway. I will see if I can use the tools with VmWare server as well).
http://www.vmware.com/support/developer/vmci-sdk/
Interesting stuff. I am still a big fan of VmWare, and I am following Windows Server Virtulization with great interest. I have not been impressed with XEN at all. The direct experience I have had with it, and the experience of others has really left a sour taste in my mouth. Granted people have gotten it to work, and it does a good job. However it takes an extensive amount of effort to deploy and is very limited in what it delivers out of the box. Virtulization for me is an enabler to allow better use of my resources. Its not an end in and of itself. I want to spend the minimal amount of time possible to deploy the solution. Right now, VmWare Server has proven to be the solution that allows me incredibly rapid deployment and flexibility. Virtual Server is also quite nice, and offers a number of additonal features for Windows guests. Both of these are stable mature products that just work, and I use them because of that.
The automation capabilities of Virtual Server seem to be quite mature (for example see the Microsoft Virtual Server Script Center). It was rather trivial for me to automate the creation of VmWare machines as well. The next step would be moving to use the CIM SDK. I may mess around with that later this week.
I have created a new web page for the SoCal Linux community. Please see http://www.socallugs.com/
The goal is to serve as an aggregation point for material presented at various user group meetings in SoCal. This makes a number of things easier for the community, and serves as a single source for captured knowledge.
I have now swapped my servers. The 32bit serves as the combined Mythtv front end/back end. The 64 bit machine is my Development, Production,QA, Evaluation VMWARE server.
I will be working on my hack shack project from here on out with a much better core infrastructure setup.
In the interest of time and performance, I have installed vmware tools by hand into my various virtual machines. When you run as many VM’s on one machine as I do (currently 15) you see performance degrade rapidly without the tools installed. System loads in the high teens and low twenties are not uncommon. That has been fixed now and I can run my machines without the nasty performance issues.
Next on the agenda is backups and monitoring. Thats been a recurring theme and every time I sit down to work on it, I get further along. Hopefully I can get backups deployed and working today. Will post with progress throughout the day.
Lately I have been building out a vm farm on my mythtv system. However the heavy IO and CPU requirements of mythv and vmware make for a poor experience with both. After thinking over the various options, I have decided on the following plan
I think that about covers the migration plan.
I have been attempting to get Active Directory authentication to work on my test virtual machine. I followed this howto and made some tweaks which resulted in the following configuration file (/etc/samba/smb.conf)
#
# Production SAMBA configuration to join my Linux systems to WYBLE.HOME
#
#======================= Global Settings =======================
[global]
## Browsing/Identification ###
# Change this to the workgroup/NT-domain name your Samba server will part of
workgroup = WYBLE
####### Authentication #######
obey pam restrictions = yes
invalid users = root
############ Misc ############
socket options = TCP_NODELAY
domain master = no
local master = no
preferred master = no
os level = 0
domain logons = no
###### Active Directory Settings #####
security = ads
realm = WYBLE.HOME
password server = 10.7.9.31
workgroup = WYBLE
# winbind separator = +
idmap backend = rid:WYBLE=2000-100000000
idmap uid = 2000-1000000
idmap gid = 2000-1000000
winbind enum users = no
winbind enum groups = no
template homedir = /home/%D/%U
template shell = /bin/bash
encrypt passwords = true
winbind use default domain = yes
restrict anonymous = 2
#some work arounds
client schannel = no
allow trusted domains = no
client lanman auth = no
client plaintext auth = no
passdb backend = smbpasswd
log level = 10
I am able to
However doing a getent passwd fails.
At first I was getting UID Range full errors. I then switched to RID IDMAP and those went away. I will keep debugging and see what I come up with.
Yesterday MIke and I purchased bicycles. We went out for a short ride around the neighborhood, and Mike discovered a number of issues with his bike. Mine was pretty good. Just a loose kickstand. I purchased an accessory kit (contained water bottle/clip, hand held tire pump, bike computer). Hopefully there will be a way to get the data off of the computer. Might have to do some hardware hacking so I can hookup serial leads. Should be a fun project.
So I plan to start logging my rides and other exercise, and posting the details. Doing a quick search on Freshmeat, turned up a couple good programs:
http://freshmeat.net/projects/sportstracker/ (Sports tracker)
http://freshmeat.net/projects/4sports/ (4 sports)
So the bike purchase and some other errands took up most of my time yesterday.
