Commit 17dc7a22 authored by Russ Fish's avatar Russ Fish
Browse files

Clean up the WinXP image build notes a little bit.

parent c91f644a
# Directions for setting up an XP image from scratch.
# These are raw notes and commands to paste into a shell on Windows.
# Further-indented stuff is optional, for checking or debugging.
# These are raw notes and commands to paste into a shell.
# Mostly Bash shell commands for Windows, some tcsh commands for Boss or Ops.
# Some (most?) of it could be scriptified with some work.
# Notice that this file has spaces instead of tabs at the beginning of lines.
# A tab in either Bash or tcsh causes it to display all of the possible command completions!
# By convention, "informational" commands are indented a couple of spaces more.
## Debugging and problem-solving stuff is double-commented.
alias v 'ls -lsF' # "Verbose" listing
setenv en emulab.net
alias rootpc 'sudo ssh pc\!^.$en \!:2*'
alias rootrd 'rd -K -g 1280x1024 -u root pc\!^.$en &'
. Start with a clean XP image, without all of the freight from Russ C's work.
......@@ -14,6 +22,7 @@ alias rootpc 'sudo ssh pc\!^.$en \!:2*'
- Set the Windows "w32time" NTP client to connect to the Emulab NTP host.
. Runs as a service, periodically contacts the time server.
ntp1 is a DNS alias for Ops.
Need to restart w32time before it sees the setsntp configuration. (?)
net time /querysntp
net stop w32time
......@@ -27,6 +36,20 @@ alias rootpc 'sudo ssh pc\!^.$en \!:2*'
cat /etc/ntp.drift
echo 0.000 > /etc/ntp.drift
- Disable the Messenger Service to keep annoying pop-ups away.
regtool -v list /HKLM/SYSTEM/CurrentControlSet/Services/Messenger
regtool -v set /HKLM/SYSTEM/CurrentControlSet/Services/Messenger/Start 4
- Disable the SSDP Discovery Service and Universal Plug and Play Device Host.
This closes port 5000 to attacks. Also the Remote Registry service.
regtool -v list /HKLM/SYSTEM/CurrentControlSet/Services/SSDPSRV
regtool -v list /HKLM/SYSTEM/CurrentControlSet/Services/upnphost
regtool -v list /HKLM/SYSTEM/CurrentControlSet/Services/RemoteRegistry
# (4 is Disabled, 3 is Manual, 2 is Automatic, 1 is only used for System services.)
regtool -v set /HKLM/SYSTEM/CurrentControlSet/Services/SSDPSRV/Start 4
regtool -v set /HKLM/SYSTEM/CurrentControlSet/Services/upnphost/Start 4
regtool -v set /HKLM/SYSTEM/CurrentControlSet/Services/RemoteRegistry/Start 4
- Set the workgroup name to EMULAB in Control Panel/System/Computer Name/Change...
No need to reboot yet.
......@@ -76,10 +99,8 @@ alias rootpc 'sudo ssh pc\!^.$en \!:2*'
mv /home{,.orig}
ln -s /cygdrive/c/Documents\ and\ Settings/ /home
. Symlink the Windows hosts file.
. Symlink the Windows hosts file into the Cygwin /etc.
ln -s /cygdrive/c/WINDOWS/system32/drivers/etc/hosts /etc/hosts
### Patch until liblocsetup HOSTSFILE is updated.
cp /etc/hosts /cygdrive/c/WINDOWS/system32/drivers/etc/hosts
. Create a proper group file. Make wheel an alias for Administrators.
mkgroup -l | \
......@@ -127,9 +148,11 @@ q
ls -l /etc/sshd_config
# Check.
grep LogLevel /etc/sshd_config
# Make it writable to edit, then change it back.
chmod g+w /etc/sshd_config
nano /etc/sshd_config
chmod g-w /etc/sshd_config
# Get a running sshd to read the config file with SIGHUP.
kill -HUP `cat /var/run/sshd.pid`
- Start sshd.
......@@ -143,16 +166,18 @@ daFluxGroup
daFluxGroup
mkdir ~root/.ssh
chown root.wheel ~root/.ssh
# On boss.
# [On boss.]
set pc=73
set ssh_args='-o "StrictHostKeyChecking no" -o "UserKnownHostsFile /dev/null"'
# This password isn't used for anything else, and doesn't need to be
# very secure because all users are in the Administrators group on the node.
eval sudo ssh "$ssh_args" root@pc$pc id
daFluxGroup
eval sudo scp "$ssh_args" ~root/.ssh/{id_dsa,identity}.pub root@pc$pc":".ssh
daFluxGroup
eval sudo ssh "$ssh_args" root@pc$pc
daFluxGroup
# On the target.
# [On the target.]
id
cd ~root/.ssh
cat {id_dsa,identity}.pub > authorized_keys
......@@ -164,20 +189,21 @@ daFluxGroup
cp -p /home/root/.ssh/authorized_keys /sshkeys/root
ls -lR /sshkeys/root
exit
# Check back on Boss.
# [Check back on Boss.]
eval sudo ssh "$ssh_args" pc$pc id
# The following will likely complain due to nonstandard host keys.
rootpc $pc id
- Install the standard host keys, dated Jun 21 2001.
ls -l /etc/ssh*
# On boss.
# [On boss.]
set pc=136
set ssh_args='-o "StrictHostKeyChecking no" -o "UserKnownHostsFile /dev/null"'
eval sudo scp -rp "$ssh_args" /proj/testbed/fish/elab-host-keys root@pc$pc":"
eval sudo ssh "$ssh_args" root@pc$pc
# On the target.
# [On the target.]
ls -l ~/elab-host-keys
ls -l /etc/ssh*key*
ls -l /etc/orig-ssh-keys
......@@ -193,14 +219,17 @@ daFluxGroup
ls -l /etc/ssh*key*
# The following should no longer complain due to nonstandard host keys.
# [On Boss.]
rootpc $pc id
- Install tools: WinZip and Emacs.
# On boss:
# [On boss:]
sudo scp -rp /share/windows/emacs-21.3-fullbin-i386.tar.gz root@pc$pc":"/tmp
sudo scp -rp /share/windows/winzip90.exe root@pc$pc":"/tmp
# Log in as root via RDP.
rootrd $pc
# [On the node, as root.]
# Graphical installer. Start with WinZip Classic, custon setup, no desktop icon.
/tmp/winzip90.exe
......@@ -209,29 +238,31 @@ daFluxGroup
tar xfz /tmp/emacs-21.3-fullbin-i386.tar.gz
# Graphical, set up the registry, start menu, etc.
C:/emacs-21.3/bin/addpm.exe
Copy the Emacs shortcut to the All Users/Desktop folder.
# Then copy the Emacs shortcut to the All Users/Desktop folder.
# Make "emacs" be the NTEmacs runemacs starter, with "emacs-exe" for a compiler.
ln -s /cygdrive/c/emacs-21.3/bin/runemacs.exe /usr/local/bin/emacs
ln -s /cygdrive/c/emacs-21.3/bin/emacs.exe /usr/local/bin/emacs-exe
- Get other stuff that "make client" depends on.
# Include files for mysql and the Boost Graph Library.
cd $uli
## Collect the include files for mysql and the Boost Graph Library.
cd /usr/local/include
tar cfz /share/windows/mysql-include.tgz mysql
tar cfz /share/windows/boost-include.tgz boost
# On Boss.
# [On Boss.]
sudo scp -rp /share/windows/{mysql,boost}-include.tgz root@pc$pc":"/tmp
sudo scp -rp /share/windows/{WSName,addusers,usrtogrp,setx}.exe root@pc$pc":"/tmp
# On the target.
uli=/usr/local/include
mkdir $uli
cd $uli
# [On the target.]
mkdir /usr/local/include
cd /usr/local/include
tar xfz /tmp/mysql-include.tgz
tar xfz /tmp/boost-include.tgz
# Build Elvin libs with GCC for testbed client programs.
# [On Boss.]
sudo scp -p /usr/testbed/www/distributions/*elvin*-4.0.3.tar.gz root@pc$pc":"/tmp
# (Need a path without embedded spaces for the make actions to work.)
# [On the node.]
# Need a path without embedded spaces for the make actions to work.
mkdir C:/elvin
cd C:/elvin
# Don't worry about a plethora of "Cannot change ownership" warnings.
......@@ -253,7 +284,7 @@ daFluxGroup
tail install.log1
make clean
SKIP[
# SKIP[
# Build Elvin for Windows on Coke, and tar it up for later installation.
scp -p bos:"/usr/testbed/www/distributions/*elvin*-4.0.3.tar.gz" /tmp
mkdir C:/elvin
......@@ -291,14 +322,14 @@ SKIP[
tar cfz /tmp/elvin4-windows.tar.gz -C /cygdrive/c Program\ Files/elvin4
scp -p /tmp/elvin4-windows.tar.gz ops:/share/windows
SKIP]
# SKIP]
# Install the Windows Elvin, built on Coke above.
# On Boss.
# [On Boss.]
sudo scp -p /share/windows/elvin4-windows.tar.gz root@pc$pc":"/tmp
sudo scp -p /share/windows/elvind.conf.windows root@pc$pc":"/tmp/elvind.conf
# On the experiment node as root (Bash shell):
# [On the experiment node as root (Bash shell):]
rootpc $pc
cd C:
ls -ld Program\ Files/elvin*
......@@ -322,54 +353,60 @@ SKIP]
ls -l /usr/local/etc/elvind.conf
"$elvin" -c `cygpath -w /usr/local/etc/elvind.conf`
## Testing: start elvinsvc from the Services Manager now.
# Make elvinsvc automatic in services manager.
# Make elvinsvc automatic in services manager, or use these commands:
regtool -v list /HKLM/SYSTEM/CurrentControlSet/Services/elvinsvc.exe
# (4 is Disabled, 3 is Manual, 2 is Automatic, 1 is only used for System services.)
regtool -v set /HKLM/SYSTEM/CurrentControlSet/Services/elvinsvc.exe/Start 2
## Remove "&& ! WINDOWS()" from $ftcc/rc.bootsetup.
# SKIP[
## Use any Windows experiment with a Program object in it for testing.
pid=testbed eid=Windows-1
pid=testbed eid=Windows-1b
pid=testbed eid=Windows-1c
$BINDIR/evproxy -s event-server -e $pid/$eid
## Uncomment rc.progagent in $ftcc/rc.config
## program-agent debugging.
ps -Welf | grep program-agent
$rc/rc.progagent shutdown
$rc/rc.progagent boot
## Debugging.
tail $LOGDIR/progagent.debug
# Could not become user: fish
# setuid(pw->pw_uid) $ftep/program-agent.c:560.
program-agent -d -e $pid/$eid -s localhost -c /var/emulab/boot/progagents
# On ops.
# [On ops.]
tevc -e testbed/Windows-1c now prog0 start \
COMMAND="bash -c 'date; hostname' > /tmp/host.txt"
# On the node.
# [On the node.]
tail /tmp/host.txt
cat /local/logs/prog0.status
# Root must have these rights: Create a token object; Replace a
# process level token; and Increase Quota rights.
# http://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-switch,
# http://msdn.microsoft.com/library/en-us/secauthz/security/authorization_constants.asp
editrights -u root -l
editrights -u root -a SeCreateTokenPrivilege -l
editrights -u root -a SeAssignPrimaryTokenPrivilege -l
editrights -u root -a SeIncreaseQuotaPrivilege -l
# program-agent
# C:\cygwin\bin\tcsh.exe (2504): *** couldn't create window, Win32 error 5
# See http://comments.gmane.org/gmane.os.cygwin.patches/2559
# This is at cygwin-1.5.17-1-winsup/cygwin/window.cc:wininfo::winthread():96
## C:\cygwin\bin\tcsh.exe (2504): *** couldn't create window, Win32 error 5
## See http://comments.gmane.org/gmane.os.cygwin.patches/2559
## This is at cygwin-1.5.17-1-winsup/cygwin/window.cc:wininfo::winthread():96
## Try starting rc.progagent as a separate service with -i for a desktop.
## Started up and stopped immediately. Needs something else in rc.bootsetup.
--dep elvinsvc.exe \
## Depend on EmulabStartup (rc.bootsetup), which depends on the elvin service,
## and also starts evproxy. But it stops rather than staying running...
--dep EmulabStartup \
## Make it manual, and explicitly start it after rc.bootsetup in EmulabStartup.
## Works, but stays in "starting" state, err in bootsetup.log:
## cygrunsrv: Error starting a service: QueryServiceStatus: Win32 error 1053:
## The service did not respond to the start or control request in a timely fashion.
# SKIP]
# For setuid() to work, Root must have these rights: Create a token object; Replace a
# process level token; and Increase Quota rights.
# http://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-switch,
# http://msdn.microsoft.com/library/en-us/secauthz/security/authorization_constants.asp
editrights -u root -l
editrights -u root -a SeCreateTokenPrivilege -l
editrights -u root -a SeAssignPrimaryTokenPrivilege -l
editrights -u root -a SeIncreaseQuotaPrivilege -l
# program-agent service start-up.
cygrunsrv -R ProgAgent
cygrunsrv -I ProgAgent -d "Emulab Program Agent" -i -p /cygdrive/c/cygwin/bin/bash \
--type manual \
......@@ -383,6 +420,7 @@ SKIP]
touch /var/log/{program-agent,ProgAgent}.log
chmod 777 /var/log/{program-agent,ProgAgent}.log
# SKIP[
# Little problem: "Must be root to run this script!"
# Add this:
# This runs as a separate Local System service on XP. Change to root.
......@@ -400,26 +438,28 @@ SKIP]
tevc -e testbed/Windows-1 now prog0 run COMMAND='ls -l /users/fish'
tevc -e testbed/Windows-1 now prog0 run COMMAND='ls -l /proj/testbed/fish'
# On the node.
# [On the node.]
cat /local/logs/prog0.status
cat /local/logs/prog0.err
cat /local/logs/prog0.out
# SKIP]
- Get the testbed client code via CVS, build, and install it.
rootpc $pc
# Start an agent and go get my Flux DSA/Bas(CVS) key.
# [As root, on the node.]
set ws_login=fish@kzin.flux.utah.edu
# Start an agent and go to your workstation to get your ssh keys for the cvs server.
eval `ssh-agent -s`
ssh-add -l
ssh -A fish@kzin.flux.utah.edu
ssh -A $ws_login
ssh-add -l
kdsa
exit
ssh -v fish@cvs.flux.utah.edu id
ssh -v $ws_login id
mkdir ~/flux
cd ~/flux
export CVSROOT=fish@cvs.flux.utah.edu:/usr/flux/CVS CVS_RSH=ssh
export CVSROOT=$ws_login:/usr/flux/CVS CVS_RSH=ssh
# First time only
mkdir CVS; touch CVS/Entries; echo . > CVS/Repository
......@@ -435,7 +475,7 @@ SKIP]
cp -p testbed/tmcd/cygwinxp/cygwin.root.bashrc ~root/.bashrc
cp -p testbed/tmcd/cygwinxp/cygwin.root.bash_profile ~root/.bash_profile
cp -p testbed/tmcd/cygwinxp/cygwin.root.emacs ~root/.emacs
# No HOME envar set for root desktop, Emacs defaults it to C:/ .
# No HOME envar is set for root's desktop, so Emacs defaults it to C:/ .
cp -p ~root/.emacs C:/.emacs
# Site-lisp files for Emacs.
......@@ -446,17 +486,16 @@ SKIP]
cat /etc/resolv.conf
cp -p ~/flux/testbed/tmcd/cygwinxp/resolv.conf /etc/resolv.conf
>> # Back-rev program_agent.c until I fix Tim's new additions for Cygwin.
cvs update -r 1.11 testbed/event/program-agent/program-agent.c
mkdir /usr/local/man/man8
# Get the binary programs into the source tree for install.
ls -l ~/flux/testbed/tmcd/cygwinxp/*.exe
# On boss:
# [On boss:]
sudo scp -rp /share/windows/{WSName,addusers,usrtogrp,setx}.exe root@pc$pc":"/tmp
# Back on the client:
# [Back on the client:]
cp -p /tmp/{WSName,addusers,usrtogrp,setx}.exe ~/flux/testbed/tmcd/cygwinxp
# Finally ready to do the Emulab makes!
mkdir ~/flux/obj-real
cd ~/flux/obj-real
v configure.trace*
......@@ -468,7 +507,7 @@ SKIP]
# The first make fails with "Cannot change ownership" warnings unpacking tg2.0 .
make client-install >& make.log1
tail make.log1
# Patch it explicitly, since the patch action gets skipped.
# No worries. Patch it explicitly, since the patch action gets skipped.
(cd ~/flux/testbed/event/trafgen; patch -p0 < tg.patch)
# If this is an update, evproxy is run by rc.bootsetup and nothing stops it.
......@@ -489,16 +528,29 @@ SKIP]
. Patch the /etc/profile file to use /home dirs if the /users mounts are down.
Remember that /etc/profile may get stepped on when you upgrade CygWin!
diff /etc/profile ~/flux/testbed/tmcd/cygwinxp
(cd ~/flux; cvs update testbed/tmcd/cygwinxp/profile)
diff /etc/profile ~/flux/testbed/tmcd/cygwinxp
# If the diffs are right, just copy the Emulab one.
cp ~/flux/testbed/tmcd/cygwinxp/profile /etc
# Otherwise, edit the file.
ed /etc/profile
/^# If the home directory doesn't exist, create it./,/^if \[ ! -d "\${HOME}" \]; then/p
/^# If the home directory doesn't exist, create it./,/^if \[ ! -d "\${HOME}" \]; then/c
### Use a local dir under sshd if the mount failed.
if [ ! -d "$HOME" ]; then
HOME=/home/$USER
fi
# If the home directory doesn't exist, create it.
if [ ]; then
###if [ ! -d "${HOME}" ]; then
.
.-10,.+5p
. Set up the tbshutdown script to run as a service, to get a shutdown signal.
editrights -u root -l
editrights -u root -a SeServiceLogonRight -l
# Don't forget to set the root password to this.
rootpwd='daFluxGroup'
###cygrunsrv -R EmulabShutdown
# EmulabShutdown is started manually later on from rc.cygwinxp .
echo "$rootpwd"
cygrunsrv -R EmulabShutdown
......@@ -506,15 +558,17 @@ SKIP]
--shutdown --type manual \
-a "--norc --noprofile -c '/usr/local/etc/emulab/tbshutdown'"
# If you see the following, try running rc.accounts or rc.bootsetup below to
# clear it up. Haven't figured this out yet...
##cygrunsrv: Error installing a service: CreateService: Win32 error 1057:
##The account name is invalid or does not exist, or the password is invalid
##for the account name specified.
# If you see this, try running rc.accounts or rc.bootsetup below to clear it up.
touch /var/log/EmulabShutdown.log
chmod 666 /var/log/EmulabShutdown.log
regtool -v list /HKLM/SYSTEM/CurrentControlSet/Services/EmulabShutdown/Parameters
cygrunsrv -Q EmulabShutdown
# Manual start-up.
cygrunsrv -S EmulabShutdown
. See if rc.bootsetup works.
......@@ -524,12 +578,12 @@ SKIP]
touch /var/log/EmulabStartup.log
chmod 666 /var/log/EmulabStartup.log
tmcc nodeid
# Missing /etc/resolv.conf .
## Missing /etc/resolv.conf .
tmcc -d nodeid
nodeid
/usr/local/etc/emulab/tmcc.bin -d nodeid
Connection to TMCD refused. Waiting ...
# Should reboot, the first time.
## Should reboot, the first time, when it changes the node ID.
$rc/rc.cygwin
v -d /sshkeys
mkdir /sshkeys
......@@ -552,7 +606,7 @@ SKIP]
regtool -v list /HKLM/SYSTEM/CurrentControlSet/Services/EmulabStartup/Parameters
sc query EmulabStartup
. Make a $HOME envar for everybody so Emacs works on startup.
. Make a $HOME envar for everybody, so Emacs works on startup from the desktop.
- Set a user environment variable: HOME = /users/%USERNAME%
- Stored in HKCU/Environment, which is HKU/*/Environment based on the user SIDs.
- The user registry key (folder) is created at first login, doesn't exist before that.
......@@ -570,12 +624,11 @@ SKIP]
================================================================
Making images
. Make the final image from a pc600.
Uninstall the experimental net devices in Computer Management/Device Manager.
Select non-control net interface, hit delete, enter. Takes about 15 seconds per interface.
This is so it will configure properly on both 600's and 850's on the way back up.
. Uninstall the experimental net devices in Computer Management/Device Manager.
# Check which one is the control net interface.
ipconfig /all
Select a non-control net interface, hit delete, enter.
Takes about 15 seconds per interface.
. Run prepare to clear out experiment-specific state.
rootpc $pc
......@@ -586,8 +639,11 @@ Making images
. Add an entry at the beginning of xpimage-log.txt, and create the image descriptor.
. Capture the image with imagezip. (How come wap create_image doesn't work?)
# On boss.
. Capture the image with imagezip.
You can specify the PC from which to grab the image when you create an image-id.
When updating existing images, I do it by hand in two stages, as below.
# [On boss.]
set pc=136 image=SP1
set pc=73 image=SP0a
set pc=2 image=SP0
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment