The Linux Test Project (LTP) provides the basis for testing the lx API implementation. The project webpage is at http://linux-test-project.github.io/ and the source is available on GitHub at https://github.com/linux-test-project/ltp.git. LTP should be built and run from within an lx zone. To build -------- As root first make sure you have the tools installed: apt-get install build-essential autoconf automake git quota For 64-bit: apt-get install libc6-dev-i386 Additional prerequisites are required activate some tests: apt-get install attr-dev libaio-dev As a normal user: git clone https://github.com/linux-test-project/ltp.git cd ltp git checkout ed01f6a05c77f65cb5d1089474c9a0e2129c581a make autotools ./configure make all As root: make install The tests can be built in both a zone that has been installed with a 32-bit version of Linux and another zone that has been installed with a 64-bit version of the same release. When testing the 64-bit zone, a copy of the 32-bit build can be run in the 64-bit zone to ensure that 32-bit applications work correctly on a 64-bit install. Running the tests ----------------- The LTP source tree provides detailed documentation on using the test suite, so this readme only give a short summary. Because many of the tests are targetted at kernel functionality which does not apply to Illumos, or test capabilities which are not available from within a zone, or test system call functionality which has not yet been completed, a skip list is used during the test run to bypass tests which are known to fail. The skip list lives in this directory and is delivered on the system. It is available from within the lx zone as /native/usr/lib/brand/lx/ltp_skiplist. As new functionality is completed, the skip list should be updated to remove tests which now work. As root: cd /opt/ltp /opt/ltp/runltp -f `cat /native/usr/lib/brand/lx/ltp_tests` \ -S /native/usr/lib/brand/lx/ltp_skiplist -p >/tmp/test.log 2>&1 When the test run has finished, the results will be logged in a date/time stamped file under /opt/ltp/results. The summary at the end of the log file should show "Total Failures: 0". If not, something is wrong. Running tests for development ----------------------------- The source for the tests can be found under the testcases directory. The largest and most useful set for lx live under testcases/kernel/syscalls. For development purposes, an individual test (or tests) can be run by listing them in a command file, one per line. For example, with a command file named ~/tcmds, to run the read01 test you setup the file so it looks like this: read01 read01 You can run that specific test as follows: /opt/ltp/runltp -f ~/tcmds -p -l ~/read.log Test status ----------- This section provides a short summary of the rationale for why tests are being skipped. LTP groups tests into command files (i.e. syscalls, nptl, etc. provided with the -f option in the runltp command shown above). A complete list of the groups can be seen in LTP source tree under the runtest directory. Some of these groups are obviously not applicable when running in an lx zone. The remaining groups still need work before they can be run. The groups shown in the runltp command above are expected to work when the skip list is used. The 'syscalls' command file runs the majority of the actual system call tests which we are interested in. The following table indicates why specific subtests are being skipped. Also note that the following tests pass in a 64-bit lx zone, but fail in a zone installed with a 32-bit Linux build: mmap15, open12, openat02 and sendfile09. Legend: x = never in a zone * = fails on kvm and bare metal too # = emulation not implemented yet - = could enable with a test zone config change - access06 wants a block device x acct01 enables bsd process accounting # add_key01 # add_key02 x adjtimex01 x adjtimex02 x bdflush01 x cacheflush01 x chmod03 need PRIV_SYS_CONFIG to set sticky bit on reg file - chmod06 needs dev x chmod07 need PRIV_SYS_CONFIG to set sticky bit on reg file - chown04 needs dev - chown04_16 needs dev # clone02 # clone08 - creat06 wants to mount a ro fs x creat07 we don't behave this way for ETXTBSY x creat08 sets euid to 'nobody', loses PRIV_FILE_SETID to set sgid x execve04 we don't behave this way for ETXTBSY # fallocate01 # fallocate02 # fallocate03 x fchmod02 need PRIV_SYS_CONFIG to set sticky bit on reg file x fchmod03 need PRIV_SYS_CONFIG to set sticky bit on reg file - fchmod06 needs dev # fchown04 mounts # fchown04_16 # fcntl06 not supported on linux # fcntl06_64 # fcntl23 leases not implemented # fcntl23_64 " # fcntl24 " # fcntl24_64 " # fcntl25 " # fcntl25_64 " # fcntl26 " # fcntl26_64 " # fcntl30 # fcntl30_64 # fcntl31 setown/getown not impl # fcntl31_64 # fcntl32 F_SETLEASE not impl # fcntl32_64 # fcntl33 F_SETLEASE not impl # fcntl33_64 # fork05 asm into %fs reg - fork09 needs a swap cap of ~9GB # fork13 decided not to support this # fork14 " # ftruncate04 need a mnt with mandatory locking # ftruncate04_64 # getdents02 wrong errno on test 4 - perf. impact too high # getdents02_64 # get_mempolicy01 x getrusage03 we don't fill in the ru_maxrss field - getxattr01 need attr/xattr.h at build time - getxattr02 - getxattr03 # ioctl03 needs /dev/net/tun # io_cancel01 libaio stuff not done # io_destroy01 # io_getevents01 # io_setup01 # io_submit01 - inotify03 needs dev # fanotify01 don't have fanotify # fanotify02 # fanotify03 # fanotify04 # fanotify05 # keyctl01 no kernel keyring support - lchown03 needs to mount ro fs - lchown03_16 - linkat02 needs dev - link08 needs dev x mem01 crashme test which expects OOM killer to save the day - mkdir03 needs dev - mkdirat02 needs dev x mknod01 makes block and chr devs - mknod07 needs dev - mknodat02 needs dev # mmap13 expects "invalid access" to SIGBUS - mount01 needs dev - mount02 needs dev x mount03 mounts ext2 - mount04 needs dev x mount05 mounts ext2 x mount06 mounts ext2 # mq_notify01 # mq_notify02 # mq_open01 # mq_timedreceive01 # mq_timedsend01 # mq_unlink01 x mremap01 x mremap02 x mremap03 x mremap04 x mremap05 # msgctl12 uses MSG_STAT # msgrcv07 MSG_EXCEPT subtest - not avail. x open01 need PRIV_SYS_CONFIG to set sticky bit on reg file # open02 expects NOATIME to cause err for unpriv user # open10 setgid on sgid subdir behavior x open11 makes device # ppoll01 # process_vm_readv01 # process_vm_readv02 # process_vm_readv03 # process_vm_writev01 # process_vm_writev02 # prot_hsymlinks /proc/sys/fs/protected_hardlinks x ptrace04 not supp on our arch # ptrace05 OS-3307 # read02 checks errno for O_DIRECT # readahead01 # readahead02 # readdir21 dir perf. issue - rename11 needs dev - renameat01 needs dev - rmdir02 needs dev x sched_getparam01 assumes Linux SCHED_OTHER return value x sched_getparam02 assumes Linux SCHED_OTHER return value # sched_rr_get_interval01 # sched_rr_get_interval02 # sched_rr_get_interval03 x sched_setparam02 tries to set Linux policies x sched_setparam03 tries to set Linux policies # sched_getscheduler01 # sched_getscheduler02 x sched_setscheduler01 x sched_setaffinity01 x sched_getaffinity01 # semctl01 all pass but SEM_STAT - linux specific # semop02 last test fails errno - expensive # sendfile02 OS-3296 # sendfile02_64 " # sendfile04 " # sendfile04_64 " # sendfile06 " # sendfile06_64 " # sendfile07 " # sendfile07_64 " sendmsg01 OS-3295 - tests actually pass x setfsuid04 no real equiv. and only for NFS server x setfsuid04_16 # sgetmask01 obsolete # setgroups04_16 expects sig11 for certain err # setns01 # setns02 # setpgid02 all pass but one, expects pid 0 to be there * setregid02 fails on bare metal, expects to lookup group "nobody" which * setregid02_16 doesn't exist. it is "nogroup" on ubuntu at least # setrlimit01 all pass but one, expects to set proc limit x settimeofday01 # setxattr01 # setxattr02 # setxattr03 # shmget05 OS-3326 # splice01 # splice02 # splice03 # tee01 # tee02 # ssetmask01 obsolete x stime01 x switch01 # sync_file_range01 # sysconf01 most pass but see OS-3305 # sysctl01 the build compiled this out, # sysctl03 this syscall is basically obsolete # sysctl04 obsolete # sysctl05 # syslog01 # syslog02 # syslog03 # syslog04 # syslog05 # syslog06 # syslog07 # syslog08 # syslog09 # syslog10 # syslog11 # syslog12 # unshare01 # unshare02 - umount01 needs dev - umount02 needs dev - umount03 needs dev x ustat01 obsolete call to stat FS x ustat02 obsolete call to stat FS - utime06 needs dev - utimes01 needs dev # utimensat01 many subtests pass but see OS-3328 for the rest # vmsplice01 # vmsplice02 # perf_event_open01 # perf_event_open02