pc3000.html 5.38 KB
Newer Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
<!--
   EMULAB-COPYRIGHT
   Copyright (c) 2005 University of Utah and the Flux Group.
   All rights reserved.
  -->
<center>
<h1>The New "pc3000" Nodes</h1>
</center>

<h2>Contents</h2>
<ul>
<li> <a href="#machines">Machines and Interfaces</a>
<li> <a href="#images">Images and Kernel Support</a>
<li> <a href="#caveats">Caveats</a>
</ul>

<hr>
<a NAME="machines"></a><h2>Machines and Interfaces</h2>
<p>
The new "pc3000" machines are
<a href="http://www1.us.dell.com/content/products/productdetails.aspx/pedge_2850">
Dell PowerEdge 2850s</a>
23
with a single 3GHz processor, 2GB of RAM, and 2 10,000&nbsp;RPM 146GB SCSI disks.
24
25
26
27
28
Each has 4 available Gigabit experimental net interfaces,
however due to lack of Gb switch ports, not all are usable at Gb speeds.
Instead, they are a mix of 100Mb and 1Gb as follows:
</p><p>
<pre>
29
30
31
32
33
# of Machines	Gb ports	100Mb ports	Machine names
40		4		0		pc201-pc240
20		3		1		pc241-pc260
40		2		2		pc281-pc320
60		1		3		pc261-pc280, pc321-pc360
34
35
36
37
38
39
40
41
42
43
44
</pre>
Note that all 160 machines have at least 1 Gb interface and all can be
used for 100Mb as well.
</p><p>
There is also a 5th Gb interface on each machine, that is used to pairwise
connect adjacent machines.  For example, pc201 is connected directly to pc202
via a cross over cable.  These interfaces are not yet available, pending
further support in the resource mapper.
</p><p>
All machines have serial console lines accessible from users.emulab.net via
the "console" command or remotely via the
45
<a href="http://www.emulab.net/kb-show.php3?xref_tag=tiptunnel">tiptunnel</a> interface.
46
47
48
49
50

</p><p>
<a NAME="images"></a><h2>Images and Kernel Support</h2>
<p>
Currently, the Emulab standard FreeBSD (FBSD410-STD, FBSD54-STD),
51
52
Linux (RHL90-STD, FC4-STD, "legacy" RHL73-STD),
and Windows (WINXP-UPDATE-pc3000, WINXP-SP{0,1,2}-pc3000)
53
54
55
56
57
58
images will run on the new machines.
</p><p>
If you have built custom images based on our standard images before
Sept 2 2005, they will likely not work on the new machines due to a lack
of the correct disk driver.  You will either need to re-customize based
on the current images or modify your existing image to add the correct
59
SCSI driver.  For BSD, add the following to your kernel config file:
60
61
62
<code><pre>
  device          mpt
</code></pre>
63
For Linux, add the following to your .config:
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
<code><pre>
  CONFIG_FUSION=y
  CONFIG_FUSION_BOOT=y
  CONFIG_FUSION_MAX_SGE=40
  CONFIG_FUSION_ISENSE=m
</code></pre>
</p><p>
For Windows you will need to re-customize based on our current -pc3000
images.
</p><p>
<a NAME="caveats"></a><h2>Caveats</h2>
<p>
<ul>
<li> <b>No bandwidth delays between 100Mb and 1000Mb.</b>
     We have not yet looked at providing bandwidth delay values between
     100Mb and 1000Mb. 
<li> <b>The "direct connect" 5th interface is not usable.</b>
     As mentioned, we need modifications to the resource mapper
     and there are also security issues to be resolved.
<li> <b>Interswitch bandwidth may limit topologies in non-intuitive ways.</b>
     Since the Gb interfaces and the 100Mb interfaces are on different
     switches, and the new machines are on different switches than the old
     pc600 and pc850 machines, the bandwidth of the interswitch links can
     have a significant impact on whether an experimental topology can map
     or not.
<li> <b>No vnode characterization has been done.</b>
90
     The so-called "colocation factor" has been arbitrarily set to 75 right
91
92
93
94
95
96
97
98
99
100
101
     now.  It will likely be clamped down considerably.
<li> <b>There might be assorted failures due to scaling issues.</b>
     Experiments of over 150 machines, which were hard to do before,
     may reveal further scaling issues that will need to be addressed.
     Expect an increase in unexpected swapin failures as a result.
<li> <b>These machines are radically different than the old machines.</b>
     Since these machines are so much faster, have so much more and faster
     memory, have so much more and faster disk, results of your experiments
     on these machines could be dramatically different.  That is, if your
     applications were CPU, memory, or disk intensive.  If you are
     interested in reproducing old results, you might want to
102
     <a href="http://www.emulab.net/kb-show.php3?xref_tag=tb-set-hardware">
103
     limit your experiments to using only pc600 and pc850 nodes</a>.
104
105
106
<li> <b>Hyperthreading is available, but SMP support is
     not currently compiled in to the stock Emulab images</b> (with the
     exception of RHL73-STD).  If you want it, you
Kirk Webb's avatar
   
Kirk Webb committed
107
108
109
110
111
112
     will have to compile your own kernel with SMP enabled and create a disk 
     image.  Note that FreeBSD 4.x SMP kernels will panic on a uniprocessor
     machine, so a combined 4.x image that runs on all Emulab nodes is not
     currently possible.  We plan to upgrade our Linux and FreeBSD 5 images
     to support SMP in the near future as well as address the 4.x SMP issue
     for standard images.
113
114
115
116
117
118
119
<li> <b>The nodes run in 32-bit mode.</b>
     The processors in these machines are the Intel EM64T Xeons, but we have
     not yet created 64-bit native versions of FreeBSD, Linux or Windows.
<li> <b>The network interfaces use "interrupt throttling."</b>
     The Intel Pro1000 cards used for the experimental interfaces in
     these machines default to using interrupt throttling.  This will
     add latency to packet delivery and reception.  See the
120
     <a href="http://www.emulab.net/kb-show.php3?xref_tag=pc3000-latency">
121
     knowledge base entry</a> for how to change or disable this feature.
122
</ul>