Anonymous

Equipment/Landin: Difference between revisions

From London Hackspace Wiki

Equipment/Landin (view source)

Revision as of 19:49, 15 July 2018

8,133 bytes added ,  15 July 2018
m
added live notes
m (Where is the power button?)
m (added live notes)
Line 39: Line 39:
* [https://www.servethehome.com/lsi-sas-2008-raid-controller-hba-information/ Avago LSI SAS2008 SAS PCIe JBOD Controller] with the following ZFS disk configuration:  
* [https://www.servethehome.com/lsi-sas-2008-raid-controller-hba-information/ Avago LSI SAS2008 SAS PCIe JBOD Controller] with the following ZFS disk configuration:  
** 12-drive ([https://www.hgst.com/sites/default/files/resources/Ultrastar_A7K1000_final_DS.pdf 1TB HGST HUA721010KLA330]) single-pool RAIDZ2 (10TB usable) mounted as /peter
** 12-drive ([https://www.hgst.com/sites/default/files/resources/Ultrastar_A7K1000_final_DS.pdf 1TB HGST HUA721010KLA330]) single-pool RAIDZ2 (10TB usable) mounted as /peter
= Build Notes =
# These are the notes for the build of Landin (and its functional twin Blanton)
# HW config and notes here: https://wiki.london.hackspace.org.uk/view/Equipment/Landin
Do the right thing and install the Software RAID-1 on the two boot SSDs. [https://blog.sleeplessbeastie.eu/2013/10/04/how-to-configure-software-raid1-during-installation-process/ Install Notes here]
SSD install note: '''NO SWAP PARTITION''' (we've got 96GB of memory and the SSDs are only 120GB - make a swapfile if we really need on the ZFS array)
Note with the above, grub-install fails, so:
#  fdisk /dev/sda  (and then sdb)
# Add in a second partition that is at the front of the drives, change new partition 2 to type 4 (BIOS BOOT)
# Then chroot /target /bin/bash and grub-install /dev/sda and grub-install /dev/sdb (assuming these are the SSDs being mirrored)
# Now system works with grub installs, reboots, etc.
FYI - sda (and similarly sdb) will look like this:
  Disk /dev/sda: 111.8 GiB, 120040980480 bytes, 234455040 sectors
  Units: sectors of 1 * 512 = 512 bytes
  Sector size (logical/physical): 512 bytes / 512 bytes
  I/O size (minimum/optimal): 512 bytes / 512 bytes
  Disklabel type: gpt
  Disk identifier:
 
  Device    Start      End  Sectors  Size Type
  /dev/sda1  2048 234452991 234450944 111.8G Linux RAID
  /dev/sda2    34      2047      2014  1007K BIOS boot
=== Debian packages to install (support for either legacy multi-user commands, compilation stuff, and more):===
Please note you should add "contrib non-free" after main to the /etc/apt/sources.list for ZFS!
'''iotop htop sudo finger bsdgames ethtool* lynx elinks net-tools openssh-server sudo screen iproute resolvconf build-essential tcpdump vlan ethtool rsync git rdist bzip2 git-core less unzip curl flex bc bison netcat nmap locate vim zsh vim-scripts zfs-dkms zfsutils-linux nfs-kernel-server samba-common-bin qemu-kvm libvirt-clients libvirt-daemon-system libvirt-daemon lshw ipmitool tftpd-hpa apt-mirror smartmontools iozone3 minicom tmux mosh
'''
=== Show off dmesg===
Why can superusers only look at dmesg nowadays? It's kinda useful to see (yeah, OK, fine, everything is a security risk)
sudo sysctl kernel.dmesg_restrict=0
kernel.dmesg_restrict = 0
NOTE ABOVE - PUT IN /etc/sysctl.conf to make it permanent.
=== Installing ZFS, Setting up ZPOOL and Live Disk Swapping ===
Already setup above in the mega-apt-get command. (Legacy note)
Please note you may need to add contrib (and possibly non-free) to the /etc/apt/sources.list (!)
  apt-get install linux-headers-$(uname -r)
  apt-get install zfs-dkms zfsutils-linux
# EASY WAY TO MAKE THE ZPOOL (NOTE WHETHER YOU WANT RAIDZ1/Z2/Z3 and the WORKING DIRECTORY)
# Note you're using -f because you're using the whole disk and ignoring legacy disklabels...
  cd /dev/disk/by-id
  sudo zpool create -f peter raidz2  `ls ata-HITACHI*|grep -v part`
(this is easy because all of the donated 1TB drives are same-model HITACHI)
# FYI - a Similar pool creation expanded out would look like this
  sudo zpool create -f kinnaman raidz2 /dev/disk/by-id/ata-HITACHI_HUA721010KLA330_PAG06BGA /dev/disk/by-id/ata-HITACHI_HUA721010KLA330_PAG06EWA /dev/disk/by-id/ata-HITACHI_HUA721010KLA330_PAG0DJ9A /dev/disk/by-id/ata-HITACHI_HUA721010KLA330_PAJ93TMF /dev/disk/by-id/ata-HITACHI_HUA721010KLA330_PAJ9ES2F /dev/disk/by-id/ata-HITACHI_HUA721010KLA330_PAJ9GPHF /dev/disk/by-id/ata-HITACHI_HUA721010KLA330_PAJ9J1EF /dev/disk/by-id/ata-HITACHI_HUA721010KLA330_PAJ9J59F /dev/disk/by-id/ata-HITACHI_HUA721010KLA330_PAJ9N1AF /dev/disk/by-id/ata-HITACHI_HUA721010KLA330_PAJ9N2TF /dev/disk/by-id/ata-HITACHI_HUA721010KLA330_PAJ9N3EF /dev/disk/by-id/ata-HITACHI_HUA721010KLA330_PBJ76D4F
==== ZFS Disk Death - what to do ====
If a 1 or 2 disks die in the ZFS zpool, you'll want to replace them. You'll see something like a disk or two with the status UNAVAIL and the zpool state being DEGRADED.
We don't want to shut off the computer, so what to do?
* Make note of the disk ID(s) and search for those drives by doing "sudo sas2ircu 0 display | less" and skip to the affected serial number (starts with the letter P in our Hitachi examples)
* Make a note of the enclosure number and the slot number on the controller.
* Make the affected disk(s) blinkk in their slots if you have enclosures that blink properly, such as
  sas2ircu 0 locate 2:1 on #REPLACE WITH THE RIGHT SLOT AND ENCLOSURE ID, DON'T JUST REPLACE THE DRIVES IN EXAMPLE SLOTS
  sas2ircu 0 locate 2:2 on
then you'll see the blinking slot(s) and can remove those affected disks, replace the drives in the disk trays, and then reinsert.
  sas2ircu 0 locate 2:1 off
  sas2ircu 0 locate 2:2 off
# Test swapping one disk that's missing/dead for another disk:
# (you can find out the serial number of the drive that you just swapped by looking in dmesg and then poking around
# in /dev/disk/by-id/ to find the new drive
Example disk replacement (remember, use zpool status to find the old disk to replace)
  sudo zpool replace -f peter ata-HITACHI_HUA721010KLA330_PAJ9N3EF ata-HITACHI_HUA721010KLA330_PBJ7DNWE
===Proxmox setup===
# We'll probably just edit LDAP users to be in that group rather than complicate things with local-remote overlays!
# libvirt:x:113: and libvirt-qemu:x:64055:
# Remember to add LDAP users to libvirt group using inheritance connectivity (or we just make the LDAP group be the 'auth'd group'
# Installed apt-mirror and sync'd the archive from RML's server
- rsync'd various items from RichmondMakerLabs  mirror seed, updated /etc/apt/mirror.list with same URLs and updated local disk hierarchy.
# TODO:
- PHYSICAL: Move card to proper guaranteed x8 slots and confirm they are negotiating at full 5GT/s (SAS2008 and Sun Quad GBE)
- crontab zpool scrub (weekly)
- enable mail sending for daemon support
x install latest sas2ircu (https://www.broadcom.com/products/storage/host-bus-adapters/sas-9210-8i#downloads) for mgmt
- install sas2ircu-status (from somewhere else)
- install bay-identifier.sh
- label drive trays with last 4 or 6 serial number chunks
- Play with bayidentifier.sh to see whether we can easily identify the disks in the right bays
- Play with sas2ircu to see if we can get drives in certain bays to flash (useful finding failed drives to replace)
- configure smartd and other warning devices
- integrate into Hackspace infra (automatic emails, root ssh keys, etc.)
- Find rails to mount into
- Configure LACP for 4xGbE
- Export NFS to certain systems over LACP link?i
- Configure ZeD - /etc/zfs/zed.d/zed.rc Good notes here: http://louwrentius.com/the-zfs-event-daemon-on-linux.html
- Enable tftpd-hpa for TFTP booting of phones and PXE systems, etc.
x Enable apt mirroring for Local Debian/Ubuntu installs
- Documentation for generating VMs
- Mirroring latest Debian OS for VM installs
- Add MOTD:
  Welcome to LANDIN.london.hackpsace.org.uk (Debian 9)
 
            NmmdhhhhhhhhhhdmmN          This system provides:
          mmhhhhhhhhhhhhhhhhhhhhmm      VM hosting for infra & test
      NmdhhhhhhhhhhhhhhhhhhhhhhhhdmN    - ACNODE -ADMINSTUFF -BRUCE -CHOMSKY
      mhhhhhhhhhhhhh/``/hhhhhhhhhhhhhm  NFS / TFTP / SMB / OpenLDAP
    Ndhhhhhhhhhhhh/`    `/hhhhhhhhhhhhdN ZFS Volumes & Replication
  Nhhhhhhhhhhhh/`        ohhhhhhhhhhhhhN
  Ndhhhhhhhhhhss.          .shhhhhhhhhhhdN Please use CHOMSKY for your general
  dhhhhhhhhh/` .os. ``      .syhhhhhhhhhd system needs.
  hhhhhhhh/`    .ssy/          `/hhhhhhhh
  hhhhhh/`      .s/              `/hhhhhh
  hhhhhh/`              -o.      `/hhhhhh To create a new VM:
  hhhhhhhh/`          -oss.    `/hhhhhhhh 1. Make Dabley Dobbles.
  dhhhhhhhhhys.      `` .os. `/hhhhhhhhhd 2. Bootle lambda frogs
  Ndhhhhhhhhhhhs.          .sshhhhhhhhhhdN 3. Baz barrybondaboo
  Nhhhhhhhhhhhhho        `/hhhhhhhhhhhhN  4. Edit the wiki with info
    Ndhhhhhhhhhhhh/`    `/hhhhhhhhhhhhdN 
    mhhhhhhhhhhhhh/``/hhhhhhhhhhhhhm   
    NmdhhhhhhhhhhhhhhhhhhhhhhhhdmN   
        mmhhhhhhhhhhhhhhhhhhhhmm       
          NmmdhhhhhhhhhhdmmN         


=== Storage Pools ===
=== Storage Pools ===
As above, one single RAIDZ2 pool of old 1TB 7200 RPM drives known as zpool 'peter'


=== Networks ===
=== Networks ===
Retrieved from ‘https://wiki.london.hackspace.org.uk/view/Special:MobileDiff/51645’