Windows XP in Emulab

Latest change: 2006-11-16 Emulab client-side update. <NEW>

Table of Contents:

  1. Overview
  2. Differences from FreeBSD and Linux in Emulab
  3. Cygwin
  4. NtEmacs
  5. Microsoft .Net Runtime
  6. Change Log <NEW>
  7. Future Work

Overview

Microsoft Windows XP is now supported as one of the operating system types for experiment nodes in Emulab, in addition to FreeBSD and Linux. Windows 2000 is not supported.

As much as possible, we have left Windows XP "stock". Some Windows services are shut down: Messenger, SSDP Discovery Service, Universal Plug and Play Device Host, and Remote Registry. Other setting changes are described under Network config and Routing below.

Before booting the node at swap-in time, Emulab loads a fresh image of Windows XP onto the experiment nodes in parallel, using our frisbee service. Emulab software automatically configures each Windows XP node, providing the expected experiment user environment including: user accounts and Emulab SSH keys; remote home, project, and shared directories; and network connections.

The Cygwin GNU environment is provided, including Bash and TCSH shells, the C/C++, Perl and Python programming languages, and several editors including Emacs, vim, nano and ed.

All of the Emulab experiment automation and monitoring services have been ported to run on Windows/Cygwin. Sometimes the Emulab documentation shows explicit command paths, such as /usr/testbed/bin/emulab-sync. Cygwin handles both Unix and Windows-style command paths, as described below .

The Emulab web interface manages a separate Windows password in the user profile, as well as making login connections to the experiment nodes. Remote Desktop Protocol service supports Windows Desktop logins from the user's workstation screen to the experiment node. SSH and Serial Console command-line connections are also supported.

Windows XP installations are more hardware dependent than Linux or FreeBSD. At present, this Windows XP image runs on most of the Emulab pc node types, including pc600, pc850, pc3000 and pc3000w (wifi) nodes.


Differences from FreeBSD and Linux in Emulab

The biggest difference of course, is that this is Windows (!), with Cygwin layered on top, and Emulab management services added. In particular, this is Windows XP (NT 5.1), with various levels of service packs and updates (see below.)

File Sharing

The second-biggest difference is that shared directories are provided not by the NFS (Network File System) protocol, but instead by the SMB (Server Message Block) protocol, otherwise known as Windows File Sharing.

The "Client for Microsoft Networks" software contacts the SMB server, in this case Samba running on the file server known as Fs (an alias for Users.) The SMB protocol authenticates using a plain-text user name and password, encrypted as they go across the network. (These Windows Shares are then accessed by UNC paths under Cygwin mounts, described below.)

In Windows GUI programs, you can just type the UNC path into the Address bar or a file-open dialog with backslashes, e.g. \\fs\share or \\fs\<username>. User and project shares are marked "not browsable", so just \\fs shows only share.

If you want to serve files from one of your experiment nodes to others, see the section on The netbt command .

Windows Passwords

A separate Windows password is kept for use only with experiment nodes running Windows. It is presented behind-the-scenes to rdesktop for RDP logins by our Web interface under Unix, and for the Samba mount of shared directories like your home directory under an ssh login, so you don't have to type it in those cases. You will have to type it each time if you use the Microsoft RDC (Remote Desktop Connector) client program from a Windows machine.

The default Windows password is randomly generated. It's easy to change it to something easier to remember.

To see or edit your Windows password, log in to Emulab, and click Manage User Profile and then Edit Profile under User Options. You will see Windows Password fields in addition to the regular Emulab Password fields.

When you change your Windows password, you will also have to re-type it as a check. The new Windows password should propagate to the Samba server on Fs instantly, so you can swap in an experiment and log in to its Windows nodes with the new password.

If you have already swapped-in experiment nodes and change your Windows password, the account information including passwords will be updated at the next Emulab watchdog daemon isalive interval. This should be in 3 to 6 minutes.

Experiment setup for Windows nodes

All you have to do is put a line specifying a WINXP OS image in your experiment .ns file, like this:
    tb-set-node-os $node WINXP-UPDATE
The Emulab Windows XP images are no longer specific to a particular hardware type. (See the Change Log for more information.) You can explicitly specify the hardware type to run on, for example:
    tb-set-hardware $node pc3000
Currently available Windows XP images are: If you use these commands: tb-set-node-startcmd, tb-set-node-tarfiles, or tb-set-node-rpms you should read the sections on Permissions and Windows GUI programs below.

Network config

Some default Windows networking features are disabled. NetBT (NetBios over TCP) ( NetbiosOptions=2 ) and DNS auto-registration ( DisableDynamicUpdate=1 ) are disabled to allow network idle detection by the slothd service. TCP/IP address autoconfiguration is disabled ( IPAutoconfigurationEnabled=0 ) so that unswitched interfaces like the sixth NICs on the pc3000's don't get bogus Microsoft class B network 169.254.0.0 addresses assigned.

The Emulab pc850 node type has five network interfaces, and the pc3000 has six. However, the Windows ipconfig /all command only shows the configuration information for the enabled network interfaces. There will always be one enabled control net interface on the 155.98.36 network. The others are disabled if not used in your experiment. (See file /var/emulab/boot/ipconfig-cache for a full listing from boot time, including the interfaces that were later disabled.)

If you specified links or LANs in your experiment network topology , other interfaces will be enabled, with an IP address, subnet mask, and gateway that you can specify in the NS file. Notice that the Windows names of the interfaces start with Local Area Connection and have a number appended. You can't count on what this number is, since it depends on the order the NIC's are probed as Windows boots.

NOTE: Often, we have seen ipconfig report an ip address and mask of 0.0.0.0, while the TCP/IP properties dialog boxes and the netsh command show the proper values. Our startup scripts disable and re-enable the network interface in an attempt to reset this. Sometimes it doesn't work, and another reboot is done in an attempt to get the network up.

Routing

Full-blown router nodes can not run Windows, i.e. rtproto Session is not supported. However, basic routing between connected network components of your experiment topology works. The Windows command to see the routing tables is route print. The IPEnableRouter=1 registry key is set on multi-homed hosts in the experiment network, before they are rebooted to change the hostname.

rtproto Static is supported in all recent WINXP images, but not in WINXP-02-16 (2005) or before.

rtproto Static-old or rtproto Manual will work in any image.

There is more information on routing in the Routing Section of the Emulab Tutorial.

Firewalls

Emulab.net is outside the University of Utah firewall, with only a minimal outer firewall , to allow free network experimentation. Even though we have closed most low-numbered ports, your Windows XP experiment nodes are still exposed to the wider Internet on higher port numbers.

If your experiment does not require that full freedom, consider adding a control net firewall node statement to your experiment ns file. This allows you to control the security level of your connection to the Internet by a style keyword , for example:

    set fw [new Firewall $ns]
    $fw set-type ipfw2-vlan
    $fw set-style basic
NOTE: There are some problems which make swapping-in Windows nodes behind a control-net firewall take longer than they should, or fail.
The Windows Firewall software is installed in WINXP-SP2 and WINXP-UPDATE images, but disabled with the netsh firewall set opmode DISABLE command, to allow RDP and SSH access to Windows nodes. There is an XP display bug related to this: sometimes the "Security Center" that shows at RDP login indicates that the Windows Firewall is turned on. Clicking through to the firewall settings shows that it is really off.

Windows nodes boot twice

Notice that Windows reboots an extra time after being loaded onto a node during swap-in. It has to reboot after changing the node name to set up the network stack properly. Be patient, Windows XP doesn't boot quickly.

With hardware-independent (Sysprep'ed) images, the first boot is actually running Mini-Setup as well, setting up device drivers and so on.

It's best not to log in to the nodes until the experiment is fully swapped-in. (You may be able to log in briefly between the first two reboots; if you see the wrong pcXXX name, you'll know that a reboot is imminent.) You can know that the swap-in process is finished by any of these methods:

NOTE: Sometimes we have seen Windows XP fail to do the second reboot. We're working on it. If you try to login after swap-in and your Windows password isn't honored, use this command on Ops to remotely reboot the node:
    node_reboot pcxxx
If you are able to log in but your remote home directory isn't mounted, you have the additional option of executing this command on the node itself:
    /sbin/reboot
The EmulabStartup service runs /usr/local/etc/emulab/rc/rc.bootsetup, logging output to /var/log/bootsetup.log. If you're having swap-in problems and rc.bootsetup doesn't finish sending ISUP to Emulab within 10 minutes, the node will be rebooted. After a couple of reboot cycles without a ISUP, Emulab gives up on the node.

If you want to prevent swap-in failure, whether to diagnose the problem or to continue manually, you may add this line to your ns file for each Windows node:
        tb-set-node-failure-action $node "nonfatal"
(where $node is replaced with the node variable, of course.)

It will still complain if it doesn't get the ISUP signal at the end of rc.bootsetup, but the swap-in will proceed and allow you to figure out what's happening.

Login connections to Windows

You can manually start up the SSH or RDP client programs to connect and log in to nodes in your experiment, or use the console command on Ops. You will have to type your Windows Password when logging in, except for SSH when you have ssh-agent keys loaded.

Or you can set up your browser to automatically connect in one click from the Emulab web interface and pop up a connection window. Once you start swapping in an experiment, the Emulab Experiment Information page contains a table of the physical node ID and logical node name, status, and connection buttons. The captions at the top of the button columns link to pages explaining how to set up up mime-types in your browser to make the buttons work, from FreeBSD, Linux, and Windows workstations:

NOTE: If you import dot-files into Emulab that replace the system execution search path rather than add to it, you will have a problem running Windows system commands in shells. Fix this by adding /cygdrive/c/WINDOWS/system32 and /cygdrive/c/WINDOWS to your $PATH in ~/.cshrc and either ~/.bash_profile or ~/.profile. Don't worry about your home directory dot-files being shared among Windows, FreeBSD, and Linux nodes; non-existent directories in the $PATH are ignored by shells.

When new Emulab user accounts are created, the default CSH and Bash dotfiles are copied from the FreeBSD /usr/share/skel. They replace the whole $PATH rather than add to it. Then we append an Emulab-specific part that takes care of the path, conditionally adding the Windows directories on Cygwin.

NOTE: The Windows ping program has completely different option args from the Linux and FreeBSD ones, and they differ widely from each other. There is a ping package in Cygwin that is a port of the 4.3bsd ping. Its options are close to a common subset of the Linux and FreeBSD options, so it will be included in future WINXP images:
 ping [ -dfqrv ] host [ packetsize [count [ preload]]] 

You can load it yourself now using Cygwin Setup .

NOTE: There are no Cygwin ports of some other useful networking commands, such as traceroute and ifconfig -a. The Windows system equivalents are tracert and ipconfig /all.

RDP details

Here are some fine points and hints for RDP logins to remote Windows desktops:

The netbt command

The NetBT (Netbios over TCP) protocol is used to announce shared directories (folders) from one Windows machine to others. (See the Name and Session services in http://en.wikipedia.org/wiki/Netbios.)

The SMB (Server Message Block) protocol is used to actually serve files. (See http://en.wikipedia.org/wiki/Server_Message_Block.)

In Emulab, we normally disable NetBT on experiment nodes, because it chatters and messes up slothd network idle detection, and is not needed for the usual SMB mounts of /users, /proj, and /share dirs, which are served from a Samba service on fs.

However, NetBT does have to be enabled on the experiment nodes if you want to make Windows file shares between them. The netbt script sets the registry keys on the Windows network interface objects. Run it on the server nodes (the ones containing directories which you want to share) and reboot them afterwards to activate. There is an optional -r argument to reboot the node.

    Usage: netbt [-r] off|on
If you use netbt to turn on NetBT, it persists across reboots.

No reboot is necessary if you use Network Connections in the Control Panel to turn on NetBT. It takes effect immediately, but is turned off at reboot unless you do netbt on afterward as well.

ipconfig /all reports "NetBIOS over Tcpip . . . : Disabled" on interfaces where NetBT is disabled, and says nothing where NetBT is enabled.

To start sharing a directory, on the node, use the net share command, or turn on network sharing on the Sharing tab of the Properties of a directory (folder.)

The machine names for UNC paths sharing are the same as in shell prompts: pcXXX, where XXX is the machine number. These will show up in My Network Places / Entire Network / Microsoft Windows Network / Emulab once you have used them.

IP numbers can also be used in UNC paths, giving you a way to share files across experiment network links rather than the control network.

There is an Emulab-generated LMHOSTS file, to provide the usual node aliases within an experiment, but it is currently ignored even though "Enable LMHOSTS lookup" is turned on in the TCP/IP WINS settings. Try nbtstat -c and nbtstat -R to experiment with this. (See the Microsoft doc for nbtstat.

Making custom Windows OS images

Making custom Windows images is similar to doing it on the other Emulab operating systems, except that you must do a little more work to run the prepare script as user root since there are no su or sudo commands on Windows. This is optional on the other OS types, but on Windows, proper TCP/IP network setup depends on prepare being run.

Cygwin

Cygwin is GNU + Cygnus + Windows , providing Linux-like functionality at the API, command-line, and package installation levels.

Cygwin documentation

Cygwin is well documented. Here are some links to get you started:

Cygwin packages

A number of optional Cygwin packages are installed in the image due to our building and running the Emulab client software, plus some editors for convenience. These packages are currently agetty, bison, cvs, cygrunsrv, ed, file, flex, gcc, gdb, inetutils, make, minires-devel, more, nano, openssh, openssl-devel, patch, perl, perl-libwin32, psmisc, python, rpm, rsync, shutdown, sysvinit, tcsh, vim, wget, and zip.

The Cygwin command cygcheck -c lists the packages that are installed, and their current version number and status. Package-specific notes and/or documentation for installed packages are in /usr{,/share}/doc/Cygwin/*.README and /usr/share/doc/*/README files. The Cygwin package site lists the available pre-compiled packages and provides a search engine.

If you want to install more Cygwin pre-compiled packages, run the graphical installer:

    C:/Software/Cygwin/setup.exe

The Cygwin command cygcheck -l package-name lists the contents of an installed package, which may help you to make a tarfile or rpm from a package you have installed. You can then cause it to be installed automatically by Emulab into all of the nodes of your experiment. See the Tutorial for more information about installing RPM's and tarballs.

Watch out for post-install scripts in:

    /etc/postinstall/package-name.sh{,.done}

Many packages not in the Cygwin package site have also been ported to Cygwin already. Download the sources to an experiment node and try

    ./configure
    make
    make install
as usual.

SMB mounts and Samba

User home directories and other shared directories are served by fs, another alias for Ops/Users, via the SMB protocol (Server Message Block, also known as Windows File Sharing) with the Windows Client connecting to the Samba server.

UNC paths with leading double-slashes and a server name, e.g. //fs, are used to access the SMB Shares under Cygwin. Emulab then uses the Cygwin mount command to make them appear on the usual Unix paths for the Emulab shared directories: /users/<username>, /proj/<pid>, /group/<pid>/<gid>, and /share.

The Cygwin mount command lists what you could access on the Samba server, with the UNC path in the first column. Unix file permissions may further limit your access on the Samba server. Log in to Ops to investigate.

/share/windows contains Windows software. See /share/windows/README.bin for descriptions of binary packages available for installation.

In Windows GUI programs, you can just type the UNC path into the Address bar or a file-open dialog with backslashes, e.g. \\fs\share or \\fs\<username>. User and project shares are marked "not browsable", so just \\fs shows only share.

Windows limitation: There is only one protection mask for everything in a whole mount/share under SMB. It's set in the "share properties" on the server (Samba config file in this case) so chmod will do you no good across SMB.

Cygwin limitation: There is a hard-coded limit of 30 mount points in Cygwin. Cygwin uses 4 of them, and Emulab uses another 3 or 4. So some of your /users mounts will fail on Windows startup if you have more than 23 or 24 members in your project, unless they are grouped into smaller subgroups .

Cygwin arcana


NtEmacs

We don't include the Cygwin X server in our XP images to keep the bulk and complexity down. So NtEmacs 21.3 is provided instead of the Cygwin X Emacs. NtEmacs "frames" are windows on the Windows Desktop, e.g. ^X-5-2 makes another one.

The /usr/local/bin/emacs executable is a symlink to /cygdrive/c/emacs-21.3/bin/runemacs.exe, which starts up an Emacs on the desktop. This only works under RDP, since SSH logins have a null desktop.

There is also a /usr/local/bin/emacs-exe executable, a symlink to /cygdrive/c/emacs-21.3/bin/emacs.exe, which is only useful as an Emacs compiler. It could be used to run Emacs in an SSH or Serial Console login window with the -nw (no windows) flag, except that it exits with emacs: standard input is not a tty. Another thing not to try is running emacs-exe -nw in a Bash or TCSH shell on the RDP desktop. It crashed Windows XP when I tried it.


Microsoft .Net Runtime

We have not included the Microsoft .NET Framework runtime in the standard WINXP images in an attempt to keep the image size down. If you need it, it's easy to install.

The installer for the Microsoft Framework Version 1.1 Redistributable Package is downloaded to /share/windows/dotnetfx.exe. Run it to download the runtime, agreeing to the EULA. It takes a few minutes, and uses 104 meg of disk.

If you want to develop .NET code as well as run it, you will need to install the .NET Framework SDK Version 1.1 .


Change Log

2006-11-16 Emulab client-side update 2006-07-28 2006-04-04 2006-02-14 2005-11-22 2005-10-28 2005-9-29 2005-8-22

Future Work

There are still some things undone: