• 12Jan
    Author: ben Categories: Infrastructure Comments: 0

    Pile o' NICs

    I know, you love your network card. You installed Linux, the NIC was autodetected at first boot, and everything “Just Worked.” Your server has been happily providing services over the network ever since.

    But what do you really know about your network card? Is it the culprit of slower performance for your CPU-intensive application? Could you benefit from any of its advanced capabilities? Today’s network interface cards offer a number of hidden gems to the savvy administrator. In this article we’ll learn some of the most important tricks to understanding your NIC in Linux. Read more »

  • 11Nov
    Author: trent Categories: Ramblings, Security Comments: 0

    Good grief.  For those paying attention, tools like sudo and the concepts behind them have been around for a really long time.  Long enough that I can barely remember working on them, though I agree with this article that I did and it did in fact occur many years before Microsoft’s “invention” of this technology.  Microsoft, apparently, doesn’t remember or chooses not to.  Read the Groklaw article on this topic for the gory details.

    [Slashdot] [Digg] [Reddit] [del.icio.us] [Technorati] [StumbleUpon]
  • 21Apr
    Author: ned Categories: Infrastructure, Security Comments: 2

    854998249_6686eb8991_m The smart folks over at Amazon Web Services just published a new white paper titled Creating HIPAA-Compliant Medical Data Applications. I’m a strong believer that it is possible to deploy Internet applications as securely “in the cloud” as in a private data center somewhere, and vendor documentation like this goes a long way toward helping others grasp this reality.

    One weakness is that the white paper barely mentions encrypting data at rest.  Here’s their accurate but incredibly concise statement:

    HIPAA’s Privacy Rule regulations include standards regarding the encryption of all PHI in
    transmission (“in-flight”) and in storage (“at-rest”). The same data encryption mechanisms
    used in a traditional computing environment, such as a local server or a managed hosting
    server, can also be used in a virtual computing environment, such as Amazon EC2 and
    Amazon S3.

    Their blog post mentions some software libraries and commercial tools for achieving encryption at rest, but generally leaves any implementation for you to figure out. There are encryption recommendations for software developers and end users, but not for system administrators (aren’t we their key demographic?). Never fear – encrypting your data at risk is easy with Linux! There are many ways to achieve encryption of data when it is stored on disk, but whole-volume encryption is often appealing because it can be implemented completely transparently to the application.

    One of the best tools for securing your data “at-rest” while it is stored on Amazon’s Elastic Block Store (EBS) is dm-crypt. It’s already built into most modern Linux kernels, and gives you extra confidence that noone else could read your EBS volumes. Anyone who’s thinking of deploying any app that stores sensitive information (in “the cloud” or in your data center) should consider implementing dm-crypt on their Linux servers. Below are instructions for creating and using an EBS volume which is protected by dm-crypt encryption…

    Read more »

  • 04Mar
    Author: ned Categories: Infrastructure Comments: 0

    I’m really looking forward to speaking at next week’s Boulder Linux User’s Group meeting, where I’ll try to cut through some of the marketing hype and provide real-world examples of ways to use “the cloud” in a real production IT department. If you’re a Linux sysadmin in the Boulder area and haven’t yet made it to a BLUG meeting, next Thursday night is a great time to come check it out!

    Here’s the meeting information:

    Our monthly meetings are at 7:00 PM on the second Thursday of each
    month at the offices of Aztek Networks., 2477 55th St, Suite 202,
    Boulder, CO. A typical meeting consists of an hour-long talk followed
    by a raffle for books, and then a question-and-answer period.

    And a little blurb about the talk:

    Cloud Computing for the rest of us: IT Infrastructure on the
    Shoulders of Giants
    
    Cloud computing is the new "web 2.0" - everyone's got one.  We'll
    focus on Linux-specific infrastructure offerings "in the cloud" and
    how you can use them in production today.  By taking advantage of
    on-demand, pay-as-you-go Linux servers and related services, you can
    save money today while simultaneously increasing performance and
    availability.  We will get hands-on during the meeting with Linux
    cloud solutions from Amazon and RackSpace.  We'll also take a step
    back and look at architectures for scaling Linux services "in the
    cloud".

    Hope to see you there!

    [Slashdot] [Digg] [Reddit] [del.icio.us] [Technorati] [StumbleUpon]
  • 05Oct
    Author: ben Categories: Infrastructure Comments: 0

    Linux users are blessed with a plethora of useful network administration tools, not the least of which is the flexible and powerful OpenSSH suite. Most Linux admins consider using ssh to securely hop from system to system, but some of the other features are less widely known.

    File Transfers
    OpenSSH includes two tools, scp and sftp, to copy files over the network, and can be used in place of FTP in most cases. For example, to copy a file from the local system to a host called serverb, use:

    % scp file serverb:

    Similarly, to copy a file from serverb to the /tmp directory, use:

    % scp serverb:file /tmp
    ben@serverb's password:
    file                                    100%    0     0.0KB/s   00:00

    Use sftp to get or put multiple files and navigate the directory structure, much like ftp:

    % sftp serverb
    Connecting to harp...
    ben@serverb's password:
    sftp> ls
    boulder
    file
    foo
    bar
    depts.txt
    sftp> get file
    Fetching /home/ben/file to file
    sftp> put anotherfile
    Uploading anotherfile to /home/ben/anotherfile
    anotherfile                                    100%    0     0.0KB/s   00:00
    sftp>

    Public Key authentication
    Public key authentication is an underused tool that is great for convenience AND security. Public key auth is a form of two-factor authentication, requiring something you have (a key file), and something you know (a passphrase).

    In a nutshell, an OpenSSH tool called ssh-keygen is used to generate two files – a public and a private key file. The public key is distributed to all the hosts you log in TO, and the private key is on the host you log in FROM. Don’t share the private key with anybody! When the key pair is generated, a passphrase is provided to protect the file. Note that the passphrase is optional, but is strongly encouraged.

    Creating and distributing the key pair is covered widely elsewhere on the web, so I won’t repeat it here. However, a few troubleshooting steps will go along way – setting up a key pair the first few times can be frustrating! Here’s some tips if it isn’t working:

    • By default, the public key file needs to be called authorized_keys2 and live in the .ssh directory of the user.
    • The permissions of the .ssh directory must be 700, and the authorized_keys2 file should be 600.
    • By default, the permissions of the private key file must also be 600
    • Public key authentication will not work if the user’s password is not set
    • By far, the most common configuration issue is a permission problem!

    SSH Tunneling
    SSH tunneling can help with remote access to systems and ports that are typically hidden by a firewall. If, for example, you can SSH from your home to a server at work, but you cannot use remote desktop to log in to your windows system at your desk, SSH tunneling can help. Since the server running SSH is on the internal network, you can “tunnel” a connection through the SSH server to access the Windows server. In the following example, the SSH server is called serverA, and the Windows desktop is desktopA. The command connects port 4389 on the local system to port 3389 on desktopA.

    % ssh -L 4389:desktopA.yourdomain.com:3389 username@serverA.yourdomain.com

    To access desktopA, open the remote desktop program and use “localhost:4389″ for the host name. To forward multiple ports, simple add additional -L flags to the command line.

    Running port forward commands manually from the command line can be tricky, and certainly isn’t very useful for Windows clients without Cygwin. Most modern SSH clients, such as PuTTY and SecureCRT, include a graphical interface for SSH tunneling.

    Hopefully these SSH tips simplify your daily administration activities!

    [Slashdot] [Digg] [Reddit] [del.icio.us] [Technorati] [StumbleUpon]
    Tags:
  • 05Oct
    Author: ben Categories: Infrastructure Comments: 0

    At Applied Trust one of our servers has a built-in SCSI disk with a single LVM volume group and several logical volumes. It also has an Apple Xserve array attached via an LSI Logic fibre channel card. The Xserve array has a second volume group with only a single logical volume. The filesystems on all the logical volumes need to be mounted at boot for the system to work properly. Unfortunately, there seems to be some bug in RHEL4 that prevents the second volume group from being detected at boot unless the SCSI adapter module (mptscsih in this case) is inserted with initrd. The Red Hat installation did not support this configuration.

    To fix it, I added a second SCSI adapter to /etc/modprobe.conf:

    alias scsi_hostadapter1 mptscsih

    then I rebuilt initrd for the kernel we’re using:

    % sudo /sbin/mkinitrd ./initrd.img 2.6.9-5.ELsmp

    I inspected the new initrd manually to make sure the module would be loaded:

    % mkdir initrd
    % cp initrd.img initrd
    % cd !$
    % cd initrd
    % gunzip -S .img initrd.img
    % cpio -i < initrd
    cpio: dev/ram: Operation not permitted
    cpio: dev/console: Operation not permitted
    cpio: dev/tty3: Operation not permitted
    cpio: dev/null: Operation not permitted
    cpio: dev/tty1: Operation not permitted
    cpio: dev/tty4: Operation not permitted
    cpio: dev/systty: Operation not permitted
    cpio: dev/tty2: Operation not permitted
    4742 blocks
    % cat init
    #!/bin/nash
    
    mount -t proc /proc /proc
    setquiet
    echo Mounted /proc filesystem
    echo Mounting sysfs
    mount -t sysfs none /sys
    echo Creating /dev
    mount -o mode=0755 -t tmpfs none /dev
    mknod /dev/console c 5 1
    mknod /dev/null c 1 3
    mknod /dev/zero c 1 5
    mkdir /dev/pts
    mkdir /dev/shm
    echo Starting udev
    /sbin/udevstart
    echo -n "/sbin/hotplug" > /proc/sys/kernel/hotplug
    echo "Loading scsi_mod.ko module"
    insmod /lib/scsi_mod.ko
    echo "Loading sd_mod.ko module"
    insmod /lib/sd_mod.ko
    echo "Loading megaraid_mm.ko module"
    insmod /lib/megaraid_mm.ko
    echo "Loading megaraid_mbox.ko module"
    insmod /lib/megaraid_mbox.ko
    echo "Loading mptbase.ko module"
    insmod /lib/mptbase.ko
    echo "Loading mptscsih.ko module"
    insmod /lib/mptscsih.ko 
    echo "Loading dm-mod.ko module"
    insmod /lib/dm-mod.ko
    echo "Loading jbd.ko module"
    insmod /lib/jbd.ko
    echo "Loading ext3.ko module"
    insmod /lib/ext3.ko
    echo "Loading dm-mirror.ko module"
    insmod /lib/dm-mirror.ko
    echo "Loading dm-zero.ko module"
    insmod /lib/dm-zero.ko
    echo "Loading dm-snapshot.ko module"
    insmod /lib/dm-snapshot.ko
    /sbin/udevstart
    echo Making device-mapper control node
    mkdmnod
    echo Scanning logical volumes
    lvm vgscan --ignorelockingfailure
    echo Activating logical volumes
    lvm vgchange -ay --ignorelockingfailure VolGroup00
    echo Creating root device
    mkrootdev /dev/root
    umount /sys
    echo Mounting root filesystem
    mount -o defaults --ro -t ext3 /dev/root /sysroot
    mount -t tmpfs --bind /dev /sysroot/dev
    echo Switching to new root
    switchroot /sysroot
    umount /initrd/dev

    The bold section indicates that the mptscsih (and it’s dependency, mptbase) will be loaded by initrd.

    I moved the updated initrd to /boot:

    % sudo mv initrd.img /boot/initrd-2.6.9-5.ELsmp.atrust.img

    … then modified /boot/grub/grub.conf appropriately:

    # grub.conf generated by anaconda
    #
    # Note that you do not have to rerun grub after making changes to this file
    # NOTICE:  You have a /boot partition.  This means that
    #          all kernel and initrd paths are relative to /boot/, eg.
    #          root (hd0,0)
    #          kernel /vmlinuz-version ro root=/dev/VolGroup00/LogVol00
    #          initrd /initrd-version.img
    #boot=/dev/sda
    default=0
    timeout=5
    splashimage=(hd0,0)/grub/splash.xpm.gz
    hiddenmenu
    title Red Hat Enterprise Linux WS Atrust (2.6.9-5.ELsmp)
            root (hd0,0)
            kernel /vmlinuz-2.6.9-5.ELsmp ro root=/dev/VolGroup00/LogVol00 rhgb quiet
            initrd /initrd-2.6.9-5.ELsmp.atrust.img
    title Red Hat Enterprise Linux WS (2.6.9-5.ELsmp)
            root (hd0,0)
            kernel /vmlinuz-2.6.9-5.ELsmp ro root=/dev/VolGroup00/LogVol00 rhgb quiet
            initrd /initrd-2.6.9-5.ELsmp.img
    title Red Hat Enterprise Linux WS-up (2.6.9-5.EL)
            root (hd0,0)
            kernel /vmlinuz-2.6.9-5.EL ro root=/dev/VolGroup00/LogVol00 rhgb quiet
            initrd /initrd-2.6.9-5.EL.img

    Reboot for settings to take effect.

    [Slashdot] [Digg] [Reddit] [del.icio.us] [Technorati] [StumbleUpon]
    Tags: