[ESXi 3.5] PXE Boot to working ESXi 3.5 Hypervisor

Need:
- ESXi 3.5 Iso File
- http://www.accessdata.com/downloads/current_releases/imager/Imager%20Lite%202.6.1.zip to open Iso and extract Files
- 7zip, WinRar, or whatever to unzip Files (7zip Recommended)
- http://syslinux.zytor.com/wiki/index.php/PXELINUX

ESXi 3.5 Extract:
1.) Extract the install.tgz from \CDROM\Sessions 1\Track 01\CDROM.
Its the Rock Ridge (RR in Isobuster) Session. Do NOT choose the one from the ISO / Boot part!

2.) Unzip the install.tgz.

3.) Extract the VMware-VMvisor-big-3.5.0_Update_4-153875.i386.dd.bz2 from the path install\usr\lib\vmware\installer to an directory of your choice.

4.) Unzip the VMware-VMvisor-big-3.5.0_Update_4-153875.i386.dd.bz2

5.) Open the File with FTK Imager.

6.) Extract the files of in Partition4\root (4 Files ldlinux.sys, mbootc32, safeboot.c32, syslinux.cfg - which are the bootloaders)

7.) Extract the files of in Partition5\root (7 Files bindmod.tgz, boot.cfg, cim.tgz, environ.tgz, license.tgz, oem.tgz, vmkernel.tgz - which are the ESXi Program Files)

PXELinux Extract:
- com32/mboot/mboot.c32
- com32/menu/menu.c32
- core/pxelinux.0

TFTP
- Create root Folder
- Copy all files from partition5\root Section within that Folder
- Copy all files from PXELinux Extract Section within that Folder
- Create Folder "pxelinux.cfg" within the root Folder
- Create an text file "default" (with NO File Ending or such) in that folder, with following text:

default menu.c32
menu title PXE Boot VMware ESXi
timeout 100

label ESXi
menu label Boot VMware ESXi
kernel mboot.c32
append vmkernel.gz --- binmod.tgz --- environ.tgz --- cim.tgz
ipappend 2

label Hard
menu label Boot from local drive
localboot 0

- Setup your tftp Server as you normally would and PXE Boot!
(For example see my PXE Boot Bart PE Post for setting up Tftpd32 for Windows)

Taken from: http://docs.google.com/View?docid=ddcwgcd6_4fs6s7jcf

[ESXi 3.5] oem.tgz and the power behind it

So, I'm going to write a quick info about ESXis most famous oem.tgz.
The oem.tgz is an file supplied by two sources:
a) the "install" oem.tgz which does sit in the root of your install medium (like pxe tftpboot, or cdrom root)
this is ONLY used DURING the installation
b) the "installed" oem.tgz which does sit in the file install.tgz
this file gets copied during the installation and is therefore active AFTER the installation.
detailed path is in
usr\lib\vmware\installer\Core\VMware-VMvisor-big-3.5.0_Update_4-153875.i386.dd.bz2
unziped to VMware-VMvisor-big-3.5.0_Update_4-153875.i386.dd
and there in Partition 5 the oem.tgz
(AccessData FTK Imager Lite to access)

Do you want to change the oem.tgz of an running ESXi System?
Just enable SSH, use Winscp and copy your oem.tgz to the folder /bootbank and replace the old, empty oem.tgz

The oem.tgz enables you to change the way the installation works.
You can for example add drivers, or add Vendor/Device Codes to the driver table

Mostly used:
- add Vendor / Device code:
/etc/vmware/simple.map

- driver
/mod

Look up driver in ESXi:
lspci -v

Watch which device does load which driver:
lspci -p

Is an HDD recognized?
fdisk -l

Just some quick infos

[ESXi 3.5] PXE Boot to Install

Easy as Cake:
Copy these files from the install CD:
BINMOD.TGZ, CIM.TGZ, IENVIRON.TGZ, LICENSE.TGZ, MBOOT.C32, MENU.C32, OEM.TGZ and VMKERNEL.GZ into your tftpboot Directory.

Insert the normal needed pxelinux.0 File, as well as the pxelinux.cfg Folder which contains following File called "default":

default menu.c32
menu title PXE Boot VMware ESXi
timeout 100

label ESXi3.5U4 Install
kernel mboot.c32
append vmkernel.gz --- binmod.tgz --- ienviron.tgz --- cim.tgz --- oem.tgz --- license.tgz --- install.tgz
ipappend 2

label Hard
menu label Boot from local drive
localboot 0

Have fun!

Taken from: http://www.vm-help.com/esx/esx3i/ESXi_PXE_install.html

[ESXi 3.5+] Enable SSH and Telnet

1) At the console of the ESXi host, press ALT-F1 to access the console window.
2) Enter unsupported in the console and then press Enter. You will not see the text you type in.
3) If you typed in unsupported correctly, you will see the Tech Support Mode warning and a password prompt. Enter the password for the root login.
4) You should then see the prompt of ~ #. Edit the file inetd.conf (enter the command vi /etc/inetd.conf).
5) Find the lines that begins with #ssh and remove the #. Then save the file. If you're new to using vi, then move the cursor down to #ssh line and then press the Insert key. Move the cursor over one space and then hit backspace to delete the #. Then press ESC and type in :wq to save the file and exit vi. If you make a mistake, you can press the ESC key and then type it :q! to quit vi without saving the file. Note: there are two lines for SSH with ESXi 4.0 now - one for regular IP and the other for IPv6. You should
6) Once you've closed the vi editor, you can either restart the host or restart the inetd process. To restart inetd run ps | grep inetd to determine the process ID for the inetd process. The output of the command will be something like 1299 1299 busybox inetd, and the process ID is 1299. Then run kill -HUP (kill -HUP 1299 in this example) and you'll then be able to access the host via SSH.

OR

6b) /sbin/services.sh restart

Tip - with some applications like WinSCP, the default encryption cipher used is AES. If you change that to Blowfish you will likely see significantly faster transfers.

Changing the port for SSH
To change the port for SSH, edit the file /etc/services and change the SSH port listed in the file. Save the file and repeat step 6 above.

Enable Telnet
The steps are the same as with SSH, but you'll remove the # from the 2 telnet entries in /etc/inetd.conf. Enabling telnet is not recommended if security is a concern.

You can also download an oem.tgz file which will enable SSH (and FTP). Copy the file to a datastore with the VI client and then to bootbank with the command cp /vmfs/volumes//oem.tgz /bootbank/oem.tgz and then reboot.

Taken from: http://www.vm-help.com/esx40i/ESXi_enable_SSH.php

Install SunRay Server Software 4.1 on Ubuntu JeOS Server 9.04 i386 (running under ESXi 4.0)

Hi Guys,

this Guide did cost me a lot of nerves and time, but its finally complete!
The most of this stuff is ripped from this nearly perfect guide.
But problem is, I did had some difficulties setting it up and got into serious troubles about dhcp (not working), dpkg (screwed up totaly) and other nasty things.
In the end I did this installation about 4 times, but now its working, and guess what? I did do complete notes! And I'm going to share these with you! ^^.

So, thats the end of the setup:
An Ubuntu JeOS Server 9.04 i386 with Vmware Tools, running on a ESXi 4.0 machine, with an SunRay Server Software (SRSS) Version 4.1!
Now, thats somethin' ain't it?
Ok, lets rock!

Ubuntu Setup
1) Download Ubuntu-Server 9.04 i386 image ( http://www.ubuntu.com/getubuntu/download-server )
2) Upload it onto your VMWare machine
3) Setup an new VM (Custom, Ubuntu Linux(32bit!), 2(!!) NIC Cards, Dynamic (SIZE) HDD(!!!))
4) Install Ubuntu Server with JeOS (so, boot the VM, choose your Language on the inital Ubuntu Setup Screen, press F4 and chose the "Minimal Virtual Machine" and then press Install Ubuntu)
5) Just an normal installation, you can choose the "Base System" and "OpenSSH Server" as additional "apt-get"'s.
6) After installation you need to install gdm:
sudo apt-get install ubuntu-desktop
sudo apt-get install gdm
sudo /etc/init.d/gdm start
sudo dpkg-reconfigure xserver-xorg
7) Just do an quick sudo apt-get update, sudo apt-get upgrade to see wheter everything is alright.
8) Thats it, your Ubuntu Server is working!

SRSS 4.1 Setup
0) Got root?
sudo su

1) Get the required packages
apt-get install libldap-2.4-2 libmotif3 module-assistant tk8.4 tomcat5.5 sun-java6-jre
apt-get install ldap-utils dhcp3-server nscd gawk iputils-ping pdksh unzip alien libgdbm3
apt-get install libx11-6 libfreetype6 libsasl2-2 libxt6 zlib1g gdm devscripts xkb-data-legacy
apt-get install xfonts-base atftpd xfonts-100dpi xfonts-75dpi xfonts-cyrillic wget ed

1 1/2) REBOOT!
reboot

2) Download and Unzip SRSS
cd /tmp/srss
download SRSS 4.1 package from sun.com (doesn't matter, I used the SUSE Enterprise Package, should be all the same!)
unzip srss_4.1_linux.zip

3) Convert all .rpm to .deb packages
find srss_4.1 -name "SUNW*i386.rpm" -exec alien -d {} \;

4) Install the .deb packages
find . -name "sunw*deb" -exec dpkg -i {} \;

5) Apply Patches
wget -O /tmp/srss/Patch-SUNWut41-1.txt http://wiki.sun-rays.org/images/6/67/Patch-SUNWut41-1.txt
wget -O /tmp/srss/Patch-SUNWutref.txt http://wiki.sun-rays.org/images/1/15/Patch-SUNWutref.txt
wget -O /tmp/srss/Patch-SUNWkio.txt http://wiki.sun-rays.org/images/f/ff/Patch-SUNWkio.txt
cd /opt
patch -p0 # Interface, for example eth1
/opt/SUNWut/sbin/utconfig
- if asked during utconfig make sure to answer /usr/share/tomcat5.5 for the tomcat path
- everything default EXCEPT tomcat path and remote server admin on ON

18) check dhcpd.conf (if you want DHCP)
make sure that /etc/dhcp3/dhcpd.conf contains the include statements for your added subnet/interface
try: /etc/init.d/dhcp3-server restart
If its NOT working, or you get problems with permissions, build your own dhcpd.conf a new! ;-):

mv /etc/dhcpd/dhcpd.conf /etc/dhcpd/dhcpd.bak
cat /etc/opt/SUNWut/net/dhcp/SunRay-options > /etc/dhcp3/dhcpd.conf
cat /etc/opt/SUNWut/net/dhcp/SunRay-interface-eth1 >> /etc/dhcp3/dhcpd.conf

19) patch and build the required kernel modules
a) prepare for kernel module building:
m-a -t prepare

b) download and apply patches
wget -O /tmp/utadem.patch http://wiki.sun-rays.org/images/0/0c/Utadem-4.1-50.nww.patch.txt
wget -O /tmp/utio.patch http://wiki.sun-rays.org/images/5/5d/Utio-4.1-50.nww.patch.txt
wget -O /tmp/utdisk.patch http://wiki.sun-rays.org/images/e/e0/Utdisk-4.1-50.nww.patch.txt
cd /usr/src/SUNWut
patch -p0 ${PULSECONF}

# create asoundrc for pulseaudio redirection
ASOUNDRC=${HOME}/.asoundrc
echo -n > ${ASOUNDRC}
echo "pcm.!default { " >> ${ASOUNDRC}
echo " type pulse" >> ${ASOUNDRC}
echo "}" >> ${ASOUNDRC}
echo "ctl.!default { " >> ${ASOUNDRC}
echo " type pulse" >> ${ASOUNDRC}
echo "}" >> ${ASOUNDRC}

24) link tftpd server directory
ln -s /tftpboot /var/lib

25) manually fix the Xsetup file
It seems that the /etc/opt/SUNWut/gdm/SunRayInit/helpers/Xsetup in SRSS 4.1 contains a minor,
but important bug which prevents commands like 'utwho' and 'utfwload' from working properly on some Linux distros.
To fix this script you have to manually edit it and make sure that the very first like contain the
exclamation mark (!) as shown here:

#!/bin/ksh -p

26) manually fix the finishsession file
It seems that the /opt/SUNWkio/lib/finishsession file in SRSS 4.1 contains a minor bug which prevents the
POST-execution scripts from being executed. To fix this script you have to manually edit it and make sure
that the line with the if check for "$theSessionUserDynDir" actually contains the "$" sign.

if [ -d "$theSessionUserDynDir" ]

27) finally you can start the SRSS by issuing the following commands
/etc/init.d/gdm restart
/etc/init.d/zsunray-init stop
/etc/init.d/zsunray-init start

28) get server "really" online
/opt/SUNWut/sbin/utadm -l # prints network config
/opt/SUNWut/sbin/utadm -L on # gets server online
/opt/SUNWut/sbin/utrestart # needed to get server online

29) reboot?
Can always help! You know the saying "Linux: BE ROOT! Microsoft: RE-BOOT!" ? 😉

NOTES:
17) OLD config with subnet, you should NOT use that:
/opt/SUNWut/sbin/utadm -A # Subnet for example 192.168.2.0

17) utadm working example:
/opt/SUNWut/sbin/utadm -a eth1
### Configuring /etc/nsswitch.conf
### Configuring Service information for Sun Ray
### configuring eth1 interface at subnet 192.168.128.0
Selected values for interface "eth1"
host address: 192.168.128.1
net mask: 255.255.255.0
net address: 192.168.128.0
host name: SunRayServer-eth1
net name: SunRay-eth1
first unit address: 192.168.128.16
last unit address: 192.168.128.240
auth server list: 192.168.128.1
firmware server: 192.168.128.1
router: 192.168.128.1
Accept as is? ([Y]/N): Y
### successfully setup "/etc/opt/SUNWut/net/hostname.eth1" file
Warning: IP address "192.168.128.1" appears in "/etc/hosts" with another hostname, fix? ([Y]/N): Y
### successfully setup "/etc/hosts" file
### successfully setup "/etc/opt/SUNWut/net/netmasks" file
### successfully setup "/etc/networks" file
### finished install of "eth1" interface
### Configuring firmware version for Sun Ray
All the units served by "SunRayServer" on the 192.168.128.0
network interface, running firmware other than version
"4.1_50_2008.09.25.12.37" will be upgraded at their next power-on.

### Configuring Sun Ray Logging Functions

DHCP is not currently running, should I start it? ([Y]/N): Y
### Error: unable to start dhcp services.
Please restart dhcp manually after utadm has completed.

17) working utadm example:
Configure Sun Ray Web Administration? ([y]/n)? y
Enter Apache Tomcat installation directory [/opt/apache-tomcat]: /usr/share/tomcat5.5
Enter HTTP port number [1660]: 1660
Enable secure connections? ([y]/n)? y
Enter HTTPS port number [1661]: 1661
Enter Tomcat process username [utwww]: utwww
Enable remote server administration? (y/[n])? y
Configure Sun Ray Kiosk Mode? (y/[n])? n
Configure this server for a failover group? (y/[n])? n

Update 139550-02 and 139727-01

1) download patches
cd /tmp
.... (download the patch update for the core package from [2]) http://sunsolve.sun.com/pdownload.do?target=139550-02&method=h
.... (download the uttsc update package from [3] http://sunsolve.sun.com/pdownload.do?target=139727-01&method=h
unzip 139550-02.zip
unzip 139727-01.zip

2) convert
find 139550-02 139727-01 -name "SUNW*i386.rpm" -exec alien -d {} \;

3) install
find . -name "sunw*deb" -exec dpkg -i {} \;

4) apply patches
wget -O /tmp/Patch-patch.txt http://wiki.sun-rays.org/images/6/6f/Patch-patch-139550-02.txt
wget -O /tmp/Patch-SUNWutref.txt http://wiki.sun-rays.org/images/1/15/Patch-SUNWutref.txt
cd /opt/
patch -p0 Update 139550-01

1) download patches
cd /tmp
.... (download the patch update from http://sunsolve.sun.com/pdownload.do?target=139550-01&method=h )
unzip 139550-01.zip

2) convert
find 139550-01 -name "SUNW*i386.rpm" -exec alien -d {} \;

3) install
find . -name "sunw*deb" -exec dpkg -i {} \;

4) apply patches
wget -O /tmp/Patch-patch.txt http://wiki.sun-rays.org/images/5/51/Patch-patch.txt
wget -O /tmp/Patch-SUNWutref.txt http://wiki.sun-rays.org/images/1/15/Patch-SUNWutref.txt
cd /opt/
sudo patch -p0 Installing VMWare Tools
http://chrysaor.info/?page=faq#ubuntu904_tools

Extract of the Weblog mentioned above:

Open a terminal window and execute the following command to download the script:
wget http://chrysaor.info/scripts/ubuntu904vmtools.sh
Now run the script that installs VMware Tools (Warning: it will download some files):
sudo bash ./ubuntu904vmtools.sh
When the script promps for answers, just hit Enter as defaults are OK. If you're using VMware Fusion, Server or Workstation, reboot the virtual machine and you are done.

Cleaning up

Shrink VMware Image Files on Linux Appliances
Posted in: Unix \ Linux, Vmware by Nico Maas on July 16, 2009

Works only with dynamic HDDs!

1. Backup!
2. Remove old / unused Packages
3. Remove downloaded Packages: sudo apt-get clean, Remove only old downloaded Packages: sudo apt-get autoclean
4. “ZeroOut” unused Space: cat /dev/zero > zero.dat ; sync ; sleep 1 ; sync ; rm zero.dat
5. Under GUI: “sudo vmware-toolbox” and use shrink on “/”
6. Under VIClient: Defrag

Additional:
Using Cisco Vlans and Trunks on VMWare ESXi 3.5/4

Posted in: Cisco, Network, Vmware by Nico Maas on July 24, 2009

Do you desperatly need to connect an real pc to just one VM? No problem! Here we go! All we need is an Switch with vlan Trunking. I used an Cisco c3560-poe8 for this, as it is my main work switch.

So you just need these snipets on the switch side:
( Need to do this in enable mode and conf -t! )

This is going to activate ip dhcp snooping globaly on the switch
(which means only marked ports are allowed to answer an dhcp request, thats important!)
ip dhcp snooping vlan 1-4094
no ip dhcp snooping information option
ip dhcp snooping

these are the vlans, one main, one “test” for the vm
vlan 1
name main
vlan 999
name test

normal client port
interface FastEthernet0/1
switchport access vlan 1
spanning-tree portfast

vm client port which does access vlan 999
interface FastEthernet0/3
switchport access vlan 999
spanning-tree portfast

vm server port, which does access normal vlan 1, and does trunk the rest
” ip dhcp snooping trust” means, that this port is allowed to answer dhcp requests
interface FastEthernet0/7
switchport trunk encapsulation dot1q
switchport trunk native vlan 1
switchport trunk allowed vlan 1,999
switchport mode trunk
ip dhcp snooping trust
end

Thats it!
Only thing you need to do on the vm machine is to start vi\vsphere client, go to configuration, network, create an new port group and insert the nic of the vmware machine you want to use on the switch port (here 3 on the c3560) – and set the port groups vlan id to 999. ( In my config, the eth0 is the "main" NIC and does do the normal Network Connection, eth1 is the "SUN" Port as configured above in the SRSS Part, so get the eth1 (Second NIC) into the vlan 999!)

And then, you can do pxe, dhcp, sunray, and allll you want!

So, thats it finally!
A REALLY BIG Thank you to all the authors of the howtos, guides, etc mentioned and unmentioned for helping me out. Without you, this would have been absoultly impossible! Thank you!!!

NOTES:
USB Mount:
/tmp/SUNWut/sessions/11/unit/dev/dsk/disk1 /usb

Using Cisco Vlans and Trunks on VMWare ESXi 3.5/4

Hello there,

are you one of these ppl using ESXi on a small server/pc under your desk to evaluate / develop software?
Are you one of these fitting the whole ESXi World into one machine, one one IDE / SATA drive and without any other VMWare Administrative Instances as your Vi\Vsphere Client? Are you one of these ppl that does not have one real 10 Gig Interface for each VM Machine into its server - but do desperatly need to connect an real pc to just one VM? No problem! Here we go! All we need is an Switch with vlan Trunking. I used an Cisco c3560-poe8 for this, as it is my main work switch.

So you just need these snipets on the switch side:

This is going to activate ip dhcp snooping globaly on the switch
(which means only marked ports are allowed to answer an dhcp request, thats important!)

ip dhcp snooping vlan 1-4094
no ip dhcp snooping information option
ip dhcp snooping

these are the vlans, one main, one "test" for the vm

vlan 1
name main
vlan 999
name test

normal client port

interface FastEthernet0/1
switchport access vlan 1
spanning-tree portfast

vm client port which does access vlan 999

interface FastEthernet0/3
switchport access vlan 999
spanning-tree portfast

vm server port, which does access normal vlan 1, and does trunk the rest
" ip dhcp snooping trust" means, that this port is allowed to answer dhcp requests

interface FastEthernet0/7
switchport trunk encapsulation dot1q
switchport trunk native vlan 1
switchport trunk allowed vlan 1,999
switchport mode trunk
ip dhcp snooping trust
end

Thats it!
Its that easy!
Only thing you need to do on the vm machine is to start vi\vsphere client, go to configuration, network, create an new port group and insert the nic of the vmware machine you want to use on the switch port (here 3 on the c3560) - and set the port groups vlan id to 999.

And then, you can do pxe, dhcp, sunray, and allll you want!

Cheers,

Nico

Astaro Security Gateway v7 Factory Reset

Access to SSH or console with root passwords:

/etc/init.d/factoryreset reset
No Access to SSH or root passwords are lost:

Connect a keyboard and monitor to the system.
Wait for boot prompt
Enter:
boot: default factoryreset

After that you can go to your during the installation definied IP like: https://192.168.0.1

Alternativ:

1. Boot on the ASG from a Knoppix CD
2. Login as root
3. Mount the root partition of the ASG with: mount –t ext3 /dev/sda7 /mnt
4. Go to the config on the ASG: cd /mnt/etc/wfe/conf/; vim setting
5. Change reset_pwd = 0 to reset_pwd = 1
6. Save
7. Run cd /; umount /mnt; reboot
8. Access the install IP

ESXi 3.5 on Dell Optiplex 745

- Exchange oem.tgz on root of ESXi 3.5 Install CD with new oem.tgz
- Install with CD with OEM Community Version or IC8 oem.tgz  (SATA on Normal Mode)
- Boot from Knoppix 5.1, copy OEM.tz onto hda5 / Hypervisor Partition  (SATA on  Legacy Mode)
- Boot, works!

Hints:
- Use UltraISO und BurnISO to modificiate / burn the stock ISO
- http://www.vm-help.com./esx/esx3i/customize_oem_tgz.php