Archive:Equipment/Borg

From London Hackspace Wiki
Jump to navigation Jump to search
Line 14: Line 14:
 
|origin=Donation <!-- If via pledge, please link to the completed pledge page on the wiki -->
 
|origin=Donation <!-- If via pledge, please link to the completed pledge page on the wiki -->
 
|location=Basement rack
 
|location=Basement rack
−
|maintainers=Oni, Tgreer <!-- If someone is nominated as managing the upkeep of this item, please list them here. No links please; it currently breaks the template. -->
+
|maintainers=Oni <!-- If someone is nominated as managing the upkeep of this item, please list them here. No links please; it currently breaks the template. -->
 
|template_ver=1.1 <!-- Please do not change. Used for tracking out-of-date templates -->
 
|template_ver=1.1 <!-- Please do not change. Used for tracking out-of-date templates -->
 
}}
 
}}
 
Borg is a virtual host server to be used to give members access to virtual machines in the space and ultimately replace many if not all the functions of [[Babbage]].
 
Borg is a virtual host server to be used to give members access to virtual machines in the space and ultimately replace many if not all the functions of [[Babbage]].
 +
 +
=Status=
 +
 +
Currently, BORG6 is awaiting a BIOS update to bring it into line with the other 3 nodes. It appears the latest version of scientific linux runs from a USB stick so flashing with the script from IBM should work, so long as the i386 libraries are loaded on said USB stick.
 +
 +
After flashing, the next stage will be to organise the disks in the head node and test the nodes to make sure they are all linked.
  
 
= Naming =
 
= Naming =
Line 42: Line 48:
 
*BORG5 - 4 CPUs, 128GB RAM - SN 99C5979
 
*BORG5 - 4 CPUs, 128GB RAM - SN 99C5979
 
*BORG6 - 3 CPUs, 128GB RAM
 
*BORG6 - 3 CPUs, 128GB RAM
 +
 +
BORGs one and two are lower spec and cannot be linked to the cluster (max of 4 nodes) so should be cannibalised for spares and disposed of.
  
 
= IP =
 
= IP =
Line 112: Line 120:
 
* http://blog.asiantuntijakaveri.fi/2013/09/reflashing-lsi-megaraid-sas-8708elp.html
 
* http://blog.asiantuntijakaveri.fi/2013/09/reflashing-lsi-megaraid-sas-8708elp.html
  
−
= Birth =
+
= Usage =
−
It is reasonable to say the birth of Borg has been long and difficult, concern has generally revolved around the 3 areas:
 
−
# Usage
 
−
# Energy Consumption
 
−
# Location
 
−
Each have been [https://groups.google.com/d/topic/london-hack-space/Ntwmbs1xa2A/discussion discussed on the mailing list] but for summary.
 
−
== Usage ==
 
  
 
== Energy Consumption ==
 
== Energy Consumption ==
Line 140: Line 142:
 
Borg (being a server) could add substantially to our electricity costs. To help this we can offset some of it by the retiring of other machines in the space. It has also been agreed that if it takes up more than 275W regularly then a decision about Borgs future will have to be made. If it goes over 300W we will have to re-think our strategy. These figures are a maximum and equate to about £27 per month in electricity costs. The power consumption will be regularly measured and anyone is welcome to report power consumption levels.
 
Borg (being a server) could add substantially to our electricity costs. To help this we can offset some of it by the retiring of other machines in the space. It has also been agreed that if it takes up more than 275W regularly then a decision about Borgs future will have to be made. If it goes over 300W we will have to re-think our strategy. These figures are a maximum and equate to about £27 per month in electricity costs. The power consumption will be regularly measured and anyone is welcome to report power consumption levels.
  
−
== Location ==
+
= Location =
  
 
The Rack in the basement.
 
The Rack in the basement.

Revision as of 09:30, 4 June 2015

Borg
BorgLHS.jpg
Model IBM 3950 M2
Sub-category Systems
Status Under Construction
Accessories Bits
Training requirement yes
ACnode no
Owner Hackspace
Origin Donation
Location Basement rack
Maintainers Oni

Borg is a virtual host server to be used to give members access to virtual machines in the space and ultimately replace many if not all the functions of Babbage.

Status

Currently, BORG6 is awaiting a BIOS update to bring it into line with the other 3 nodes. It appears the latest version of scientific linux runs from a USB stick so flashing with the script from IBM should work, so long as the i386 libraries are loaded on said USB stick.

After flashing, the next stage will be to organise the disks in the head node and test the nodes to make sure they are all linked.

Naming

We are naming this machine after Anita Borg. Also the Borg.

They're currently labeled as BORG1 through BORG6 from bottom to top.

Access

Currently connected to the SPC1601 PDU, we need a better way of adding accounts.

Specs

IBM 3950 M2


  • BORG1 - 3 CPUs, 8GB RAM
  • BORG2 - 3 CPUs, 8GB RAM
  • BORG3 - 4 CPUs, 128GB RAM - SN 99C5980
  • BORG4 - 4 CPUs, 128GB RAM - SN 99B3501
  • BORG5 - 4 CPUs, 128GB RAM - SN 99C5979
  • BORG6 - 3 CPUs, 128GB RAM

BORGs one and two are lower spec and cannot be linked to the cluster (max of 4 nodes) so should be cannibalised for spares and disposed of.

IP

172.31.24.11

RAS

The RAS II is a separate control system that is setup in the bios and accessible as soon as a BORG unit has power (i.e, if the green light is flashing OR solid). Using a web browser head to either

Username USERID password PASSW0RD

BIOS

With dmidecode 

* Borg3 Version A3E170AUS-1.16
* Borg5 Version 1.16
* Borg6 Version 1.11

ScaleXpandr

In order to link upto 4 borgs together we need to use the special cables. http://www.redbooks.ibm.com/redbooks/pdfs/sg247630.pdf - page 235 shows how this is done to create an SMP style set of nodes.

Logbook

Borg/LogBook

Running Debian

  • Needs the non-free bnx2 firmware on a flash drive for the install to work, (could try to add it to the install initrd)
  • Something something IBM Calgary IOMMU something something leads to DMA errors and the LSI MegaRaid raid card dosn't work, booting with "iommu=soft" makes it work but may not be ideal. search https://www.kernel.org/doc/Documentation/x86/x86_64/boot-options.txt for iommu
  • The incantation seems to be: iommu=soft,calgary megaraid_sas.msix_disable=1

Upgrading the bios

Do a diskless boot, the go do Debian and then "Jessie amd64 Diskless for BORGs", log in as root (password 'root', this diskless setup is for testing only!), then:

cd ibm-bios/z/ ./lflash64

This is an upgrade to bios version 1.16

We also need to upgrade:

The RSA II thing: https://www-947.ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5086633 The FPGA (in the scaleXpander?) The BMC

We might be able to do it individually, or perhaps use the IBM UpdateXpress thing, which needs SUSE Linux Enterprise Server 11 x86-64 or Red Hat Enterprise Linux 6 x86-64.

We need to reset all the bios etc settings to there defaults.

link dump

good megaraid cli guide:

reflashing?!?:

Usage

Energy Consumption

Recent tests comparing Borg3 with AWS , rendering 30 Frames of a small Blender Scene with Blender 2.76 and maxing out the processors:

AWS

  • $0.520 per On Demand Linux c1.xlarge Instance Hour at 12 hours
  • Cost is roughly £4.60 at current exchange
  • Spot instances are possible with Brenda or similar - at $0.07 the cost would be roughly 53p

Borg3

  • £0.12p per Kilowatt Hour
  • With all 16 Cores operating at 100% Borg was drawing 560Watts
  • To render the same scene took 206minutes
  • Total Cost 23p

The main issue of course - is it worth spending the time linking them all or just keeping one with some method of spinning it up and down?

Borg (being a server) could add substantially to our electricity costs. To help this we can offset some of it by the retiring of other machines in the space. It has also been agreed that if it takes up more than 275W regularly then a decision about Borgs future will have to be made. If it goes over 300W we will have to re-think our strategy. These figures are a maximum and equate to about £27 per month in electricity costs. The power consumption will be regularly measured and anyone is welcome to report power consumption levels.

Location

The Rack in the basement.

Potential uses

  • Rendering video and 3D
  • Bio-informatics number crunching (bio-hackers?)
  • simulation
  • realtime ray-tracing
  • Radio FFT decoding in real-time (Cubesat related)

Showing 2 related entities.

...

Retrieved from ‘https://wiki.london.hackspace.org.uk/w/index.php?title=Archive:Equipment/Borg&oldid=47175’