README 10.6 KB
Newer Older
emellor's avatar
emellor committed
1

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
----------------------------------------------------------------------

Status of xm-test

The xm-test environment was written in 2006.  It was mostly not
updated or adapted to new / changed functionality since then.
("it's slowly rotting", Keir Fraser, xen-devel mailing list,
June 2009)
During the last weeks there was the attempt to fix most of the
problems - but there are still some major issues:
o The reporting infrastructure is 'long gone' (Dan Smith, Mail from
  Aug 2009).  Therefore there is no way to send out reports. When
  running the runtests.sh you should therefore specify the '-d'
  option.
o The initrd.img (which is downloaded) is quite old and needs an
  update. Especially functionality for migration it is missing.
  [It should be discussed using an existing distribution instead of
  compiling a micro distribution and put this into the initrd.)

Andreas Florath <xen@flonatel.org>
August 2009

----------------------------------------------------------------------


emellor's avatar
emellor committed
27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52
xm-test README

Copyright (C) International Business Machines Corp., 2005
Author(s): Dan Smith <danms@us.ibm.com> 
  	   Woody Marvel <marvel@us.ibm.com>

Overview
========

This suite provides a framework for testing the Xen userspace tools.
The directory structure is:

  ./xm-test
   |
   +-/lib: Python support libraries
   |
   +-/ramdisk: Staging area for building the test ramdisk
   |
   +-/tests
   | |
   | +-/create: Tests for the 'xm create' command
   | +-/destroy: Tests for the 'xm destroy' command
   |  . . .
   |
   +-/utils: Utility scripts for ramdisk building

53 54 55 56
Reports are posted here:

   http://xmtest.dague.org

emellor's avatar
emellor committed
57 58 59 60 61 62 63 64 65

Building
========

Before the test suite can be used, the ramdisk must be built from
source.  All source needed for this process is automatically
downloaded, extracted, and compiled.  Due to the need to create
special files, this process must be done as root:

dan's avatar
dan committed
66
   # ./autogen
emellor's avatar
emellor committed
67 68 69
   # ./configure
   # make

dan's avatar
dan committed
70 71 72
NB: If you have the initrd.img from another installation of xm-test,
you can copy it into the ramdisk directory to eliminate the need to
rebuild it.  If you do this, there is no need to run 'make' again.
73
Simply copy the initrd-X.Y-ARCH.img file into ramdisk/ and then run:
74 75 76

   # make existing

77 78 79 80
Or, you can run:
   # INITRD="http://url.of.initrd.repo/" make existing

You do not need to include the name of the image itself in the url, 
81
however, an initrd with the right name (initrd.X.Y-ARCH.img) and version 
82 83 84 85
number must exist at that location.  The script will determine which 
version of the initrd it needs and try to download the right file from 
that location.

86 87 88 89 90
This will set up the link so that xm-test will use the existing
ramdisk.  Next, just run "runtest.sh" normally.  Note that in general,
you should not attempt to use a ramdisk from a previous minor version
of xm-test (i.e., don't use a ramdisk from 0.4.0 with 0.5.0.  0.5.0
should work for 0.5.3 though)
emellor's avatar
emellor committed
91

92 93 94 95

BUILDING with HVM Support
=========================

96
If you'd like to build and run this with hardware virtual machine assist
97 98
(HVM) support to test fully virtualized disk images on VMX/SVM hardware, 
please add the --enable-hvm-support option to configure:
99 100

  # ./autogen
101
  # ./configure --enable-hvm-support
102 103 104 105 106 107
  # make

The ramdisk/bin/create_disk_image script, which builds the full virt
disk.img, requires Lilo 22.7+ to be installed on the system. Lilo is 
used to install the bootloader on the disk.img.

108
If HVM support is enabled, the ramdisk/bin/create_disk_image script
109 110
will be run to create a full virt disk.img in the ramdisk directory. The
script, by default, will look in /boot for the first non-Xen kernel it
111 112
runs across. If you'd like to set xm-test to use a specific kernel,
rather than the first one it finds in /boot, you can configure it in
113 114
with the "--with-hvm-kernel=KERNEL" option:
 
115
# ./autogen
116
# ./configure --enable-hvm-support --with-hvm-kernel=KERNEL
117 118 119 120 121
# make

Otherwise, you can always rerun the create script using the -k option
to use a specific kernel. 

122 123 124 125 126 127 128
The disk.img created for HVM testing must contain a pcnet32 driver for
network tests. The ramdisk/bin/create_disk_image script will, by default,
look in the /lib/modules directory associated with the kernel being
used. If you'd like to specify a different location for the driver or
want to tell the script that the driver is built into the kernel, please
use the "--with-driver-dir=DRVDIR" configure option. If built into
the kernel, please use the key word "builtin" with the option:
129
 
130
# ./autogen
131
# ./configure --enable-hvm-support --with-driver-dir=builtin
132
- or -
133
# ./configure --enable-hvm-support --with-driver-dir=/driver/directory
134
# make
135 136
 
Xm-test will look for disk.img in the ramdisk directory when run by
137
default.
138

emellor's avatar
emellor committed
139 140 141 142

Running
=======

dan's avatar
dan committed
143 144 145 146 147
To run the full test suite, do the following as root:

   # ./runtest.sh <logfile>

This will run all tests, as well as generate and submit a report at
root's avatar
root committed
148 149 150
the end. All output files will begin with "<logfile>."
If you wish to prevent submission of a report, add "-d" to the 
command line like this:
dan's avatar
dan committed
151 152 153 154 155 156 157 158 159

   # ./runtest.sh -d <logfile>

It may be useful to run tests without submission as above, and then
submit the report at a later time.  To do so, run runtest.sh with the
-s flag and the name of the previously-generated report:

   # ./runtest.sh -s <logfile>

root's avatar
root committed
160 161 162 163 164 165 166 167
Group test sets are supported in xm-test. This is form of layering of 
tests groups/cases/tests. In the framework directory "grouptest",
files exist for group processing. The user can add groups, casenames
and test lists as required. Default group run is "grouptest/default".

   # ./runtest.sh -g <groupname> <logfile>

* NOTE: There is a quick set of tests in group mode, that was added to
168 169
run certain casenames and tests, and there is a "medium" group, which is a
medium-length run (around 20 minutes).  Neither is a substitute for the full
root's avatar
root committed
170 171
xm-test test suite.
   # ./runtest.sh -g quick <logfile>
172
   # ./runtest.sh -g medium <logfile>
dan's avatar
dan committed
173 174 175 176 177



It may be desirable to run a specific test group.  This can be
accomplished by doing the following:
emellor's avatar
emellor committed
178 179

   # cd tests/create
dan's avatar
dan committed
180 181 182 183 184 185 186
   # TEST_VERBOSE=1 make check

When developing or debugging a specific feature, a single test can be
run to avoid having to run even a whole test group:

   # cd tests/create
   # TEST_VERBOSE=1 make check TESTS=01_create_basic_pos.test
emellor's avatar
emellor committed
187 188 189 190 191 192 193

The runtest.sh script will create several files, including a .report
file, which is the cleaned up, email-friendly report of failures.
Additionally, the script will submit your results to the development
team for trend analysis.  This helps us determine the level of success
people "out there" are having with different versions of Xen.

194 195 196 197
Note: you should generally run xm-test with a minimum of memory
allocated to Dom0.  More memory available for allocation to DomUs
means a more rigorous test.

dan's avatar
dan committed
198 199 200
BIG FAT WARNING: The test framework assumes it is running on a
dedicated machine.  As such, the library automatically destroys any
running DomUs on the system to provide each test with a "clean slate".
emellor's avatar
emellor committed
201 202


Ewan Mellor's avatar
Ewan Mellor committed
203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245
Testing the XML-RPC and Xen-API interfaces of xend
==================================================

The xm-test suite can be used to test xm's interface with xend using
either XML-RPC or the Xen-API. In order to use either one of these modes,
xm needs to be configured using its configuration file
'/etc/xen/xm-config.xml'.
Note: The current default configuration after a fresh install of the xen
sources currently is to use the XML-RPC interface for communication with xend.

Example content for the xm-config.xml for using the Xen-API looks as
follows:

<xm>
  <server type='Xen-API'
          uri='http://localhost:9363/'
          username='me'
          password='mypassword' />
</xm>

This configuration makes xm talk to xend using port 9363. For this to
work, also xend needs to be configured to listen to port 9363. Therefore
The following line must be in /etc/xen/xend-config.sxp.

(xen-api-server (( 127.0.0.1:9363 none )))

To communicate via the legacy XML-RPC interface, the file
'/etc/xen/xm-config.xml' may simply have the following content or
may be complete remove from the /etc/xen directory.

<xm>
</xm>

A few tests have been written for the xm-test suite that test the
Xen-API interface directly without relying on 'xm'. These tests can be
found in the grouptest 'xapi' and for them to work properly, xm must have
been configured to use the Xen-API following the instructions above. To
run these test, the following command line can be invoked:

   # ./runtest.sh -g xapi <logfile>



emellor's avatar
emellor committed
246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264
Extending
=========

Additional tests may be added in existing groups to test additional
cases for a given xm subcommand.  Test programs should be named
according to the following scheme:
	
  XY_group_name_{pos,neg}.py

Where:
  XY        is the next number in line
  group     is the name of the subcommand being tested
  name      is the short name of the test
  {pos,neg} denotes whether this is a positive or negative test case

New subcommand groups should be added as directories named after the
subcommand itself.  The "Makefile.am.template" should be copied into
the new group directory as "Makefile.am".

dan's avatar
dan committed
265 266 267
See the Writing_Tests_HOWTO file for more detailed information on
adding tests to the suite.

emellor's avatar
emellor committed
268 269 270 271 272 273 274 275 276 277 278 279 280 281 282

Developer Notes
===============

Our library provides a DomU console abstraction for automated
execution of commands.  Please note that this is relatively fragile,
and is intended for use only with the ramdisk built by the framework.
Because the console experiences some occasional corruption, this
method is not completely perfect at the moment, although the authors
use it with relatively few problems.


Known Issues
============

283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300
If you create a domain with a small amount of memory, under 32MBs, you
may run into out of memory situations for the domain. There's no way
to know the amount of memory needed by the kernel and modules used. Xm-test
uses 64MBs as default and that should work. If there are out of memory
issues, the default can be changed. Edit xm-test/lib/XmTestLib/XenDomain.py
and change ParavirtDefaults and HVMDefaults "memory".

There are two tests that work with small memory, 15_create_smallmem_pos.py 
and 16_create_smallmem_neg.py. The first makes sure the default 32 MBs 
limit works. The second checks a low memory fail situation. These tests
are located in the xm-test/tests/create directory and can be easily edited
to change the MEM value they should test. If the 32MBs test fails, the
failure should be reported to the Xen xen-devel mailing list. The Xen
tools use 32MBs as a lower acceptable limit for domain creation. The Xen
mailing lists are located here:

http://lists.xensource.com/

emellor's avatar
emellor committed
301 302 303 304 305 306 307 308 309 310

Reporting Bugs
==============

If you find a bug in the test framework, report it to:

   Dan Smith <danms@us.ibm.com>

If you find a bug in a specific test case, contact the author of the
test case first.