CSE 265: System and Network Administration

Lab #10

Today we continue with network file services, but first consider linux kernel modules.

1. Exploring Kernel Modules

   There are four methods of modifying the configuration of a kernel: modify tunable kernel parameters; provide kernel parameters at boot time; compile the kernel from scratch; and load new drivers and modules on the fly into an existing kernel. The latter is the approach that we explore today.

   Loadable kernel modules are stored under /lib/modules/version where version is the version of your Linux kernel. Recall that uname -a will tell you which kernel you are currently running. The kernel on your RH/Fedora systems has been upgraded (leaving the old one in place) and so there will likely be multiple kernel versions. Take a look through the various subdirectories to see what modules (.ko files) are there.

   Find the vfat module, which is the driver for the Windows FAT filesystem with NT long name support. What type of device do you think the rio500 module supports? Recall that the find command can help you find the rio500.ko file, with a find . -name "*rio500.ko" -print

   The /sbin/lsmod command is used to list the currently loaded modules. Run it to see the modules loaded on your system.

   The /sbin/insmod command is used to load a kernel module. Try loading the dummy device. Can you guess what it does? How do you know that it has been loaded? Is it being used?

   The /sbin/rmmod command is used to unload a kernel module. Unload the dummy module. How do you know that it is no longer loaded? Can you unload the raid5 module?

   Now load the msdos module. You'll get several errors about unresolved symbols. This is because the msmod module has dependencies on another module, and can't be loaded until all dependencies are satisfied. To find out what symbols are a problem, run dmesg piped to tail. Look at the common prefix for each of the unresolved symbols. Go ahead and load the missing module first, and then load the msdos module next. What does the "msdos" mean in the fat line of lsmod output?

   Now unload the two modules you just added, but unload msdos last. Do you understand the error message you got? Now unload them in the correct order.

   Many modules are loaded automatically when they are needed, rather than only after being loaded manually as we have been doing so far. However, in order to load certain modules, it is important to know what modules it depends on. That information is stored in a modules dependency file called modules.dep. You can find it in the modules directory for your kernel. If you grep for msdos, you'll see the dependency that you discovered above.

   There is another utility that can automatically resolve module dependencies. It is called modprobe. It can be used to load the msdos module (along with any dependencies).

   This time use modprobe to load the dos module. Run lsmod to see which modules are listed. Is there a dos module? What was loaded instead? The command modprobe -c | grep dos will help you understand what is going on.

   Aliases are defined in /etc/modprobe.conf, where you will see that eth0 and eth1 are just aliases for the modules that know how to use the ethernet ports.

2. Installing AFS client on a Linux Machine

   Courtesy of Ashutosh Mahajan

   Fedora

   Precompiled RPM packages are available for some kernels of Fedora at [openafs.org]. For other kernels, the following steps should be followed. There are 4 parts of the whole excercise. In first part we just get the tools necessary to compile openafs. In second part we actually compile the client. In third, we install the binaries and kernel-modules. In fourth part we use the afs space. Note that the second and fourth parts do not involve any root privileges. So it is highly recommended that you undertake the steps in these parts as a normal user and not as root. Part II may be skipped by downloading openafs-client and openafs-kernel-source RPMs from openafs.org and continuing with Part III. But this is untested.

   Part I

   First we will get a new kernel and its headers so that everyone in the lab is on the same page. These steps require root privileges.
   1. Make sure you have enough space (~300MB) in your /home filesystem. (If not, you should do these exercises where you do have space, such as in /tmp.)
   2. yum install fedora-rpmdevtools
   3. yum install kernel kernel-devel to install new kernel 2.6.15-1.1833_FC4 and development tools (actually, note that this process will *not* work properly with kernel 2.6.16 which has a known bug).
   4. reboot into new 2.6.15 kernel

   Part II

   Now we have all the tools necessary to build our own RPM packages. As normal user do the following steps:
   1. cd ~
      fedora-buildrpmtree
      cd rpmbuild
      ls
   2. See different directories and their contents. Read man page of rpmbuild to understand what these are meant for.
   3. Now we will get the source files for openafs. Generally source files are just a collection of files containing source code. The openafs website bundles source code and instructions to compile them so that, after compiling, we get an rpm package which we can install like any other program. This makes administration of software more systematic. Get the source-rpm by:
      wget http://openafs.org/dl/openafs/1.4.0/fedora-4/SRPMS/openafs-1.4.0-fc4.1.src.rpm
   4. rpm -ivh openafs-1.4.0-fc4.1.src.rpm
   5. ls ~/rpmbuild/SOURCES
      Look at the source files.
   6. Now we will compile these source files to make an rpm package.
      rpmbuild -ba ~/rpmbuild/SPECS/openafs.spec
      This command reads instructions from openafs.spec and compiles the files in the source directory and builds the rpm package.
   7. See the compilation of openafs source progress. It took me around 15-20 minutes on my sandbox machine. check frequently to see if the machine has run out of memory or if its swapping. Close any unneccessary applications if needed.
   8. cd ~/rpmbuild/RPMS
      ls
   9. See the rpm files that have been compiled.

   Part III

   At this stage we have the required rpms. We will now install and configure openafs-client. These steps require root privileges.
   1. cd ~yourid/rpmbuild/RPMS/i386
   2. rpm -ivh openafs-1.4.0-1.1.i386.rpm
   3. rpm -ivh openafs-client-1.4.0-1.1.i386.rpm
      This should throw an error:
      openafs-kernel is needed by openafs-client-1.4.0-1.1.i386
   4. Note that we don't have openafs-kernel rpm file. This package provides drivers for openafs filesystem to the kernel. We will compile the modules and install them without making an rpm. First we unpack the source files.
      rpm -ivh openafs-kernel-source-1.4.0-1.1.i386.rpm
   5. cd /usr/src/openafs-kernel-1.4.0/
   6. Read README file.
   7. cd src
   8. We will now compile openafs-kernel modules. Note that this is different from the way we compiled rpm files before.
   9. ./configure
   10. make (takes 5 minutes)
   11. ls src/libafs/MODLOAD-2.6.15-1.1833_FC4-SP/*.ko
       This should show:
       libafs-2.6.15-1.1833_FC4.ko openafs.ko
   12. mkdir /lib/modules/2.6.15-1.1833_FC4/kernel/fs/openafs
   13. cp /usr/src/openafs-kernel-1.4.0/src/src/libafs/MODLOAD-2.6.15-1.1833_FC4-SP/openafs.ko /lib/modules/2.6.15-1.1833_FC4/kernel/fs/openafs
   14. make a copy of /lib/modules/2.6.15-1.1833_FC4/modules.dep
       cp /lib/modules/2.6.15-1.1833_FC4/modules.dep ~/modules.dep.bak
   15. Edit /lib/modules/2.6.15-1.1833_FC4/modules.dep
       Add a line:
       /lib/modules/2.6.15-1.1833_FC4/kernel/fs/openafs/openafs.ko:
       (somewhere near line 1342 or so, exact line number doesnt matter, save and exit)
   16. The kernel module for openafs has been installed. Now to load the module, do:
       modprobe openafs
   17. lsmod | grep openafs
       You should something like:
       openafs 554912 0
   18. Now we can install the openafs-client rpm we compiled earlier.
       cd ~yourid/rpmbuild/RPMS/i386
   19. rpm -ivh openafs-client-1.4.0-1.1.i386.rpm
   20. This should give an error similar to: openafs-kernel is needed by openafs-client-1.4.0-fc4.1.i386. Note that we installed kernel-modules manually and did not use rpm. Since we are sure that we dont need openafs-kernel, we will install openafs-client by ignoring the dependencies.
       rpm -ivh --nodeps openafs-client
   21. Installation is complete. Now we will configure the client to access AFS directories provided by Lehigh.
   22. Edit /usr/vice/etc/CellServDB
       It should have:

           >cc.lehigh.edu          #lehigh university
           128.180.39.25           #fs2.cc.lehigh.edu
           128.180.2.13            #fs3.cc.lehigh.edu
           128.180.2.10            #fs4.cc.lehigh.edu
           128.180.2.11            #fs5.cc.lehigh.edu
       

   23. edit /usr/vice/etc/ThisCell
       It should only have one line:
       
       cc.lehigh.edu
   24. see that you have directory /afs. It should have been created when we installed openafs RPMs.
   25. To start the client do
       /etc/init.d/openafs-client start
   26. You should now be able to see /afs/cc.lehigh.edu and different sub-directories therein.

   Part IV

   Using AFS: Lehigh's afs provides a lot of commercial software like matlab, maple, mathematica. For linux-machines these are located in /afs/cc.lehigh.edu/common/i386_linux24/bin. You can also see your own lehigh folder in /afs/cc.lehigh.edu/home/<your lehigh id> and also your webpage in www-data in your Lehigh-home directory. Note that you wont be able to read/execute your files. You have to log-in to do so.
   1. Install old X-windows libraries for compatibility with matlab.
      yum install xorg-x11-deprecated-libs
   2. switch back to a normal user.
   3. klog lehigh-id
      This attempts to acquire an AFS token for your lehigh-id so that you can access your files. Enter your Lehigh password.
   4. Try running matlab from the afs partition.
      cd /afs/cc.lehigh.edu/common/i386_linux24/bin
      ./matlab
      or
      ./matlab -nojvm -nosplash for non-graphical interface.
   5. Use klist to see who all have valid login tokens for afs directory. Use mount -l to see if AFS is shown as mounted.
   6. See where you have write access in /afs.

   Debian

   Debian provides a way in which modules can be compiled from source and installed as Debian Packages (like RPMs) automatically, without manual compilation, editing of configuration files etc. Here is a summary of how to do this:
   1. apt-get install openafs-client module-assistant.
   2. module-assistant update
   3. module-assistant prepare openafs-modules
   4. module-assistant auto-build openafs-modules
   5. This will download the source files and compile modules for the kernel to make debian packages which can then be installed using dpkg.
   6. The configurations files are same, but are located in /etc/openafs instead of /usr/vice/
   7. Fedora doesn't seem to have a process of automating the compilation of modules for the kernel. Switch to Debian now!

In order to sign the lab completion sheet, you will need to:

1. show me that you have full access to your AFS file system